08:01:47 <jose_lausuch> #startmeeting Functest weekly meeting February 7th 2017
08:01:47 <collabot> Meeting started Tue Feb  7 08:01:47 2017 UTC.  The chair is jose_lausuch. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:01:47 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:01:47 <collabot> The meeting name has been set to 'functest_weekly_meeting_february_7th_2017'
08:01:52 <jose_lausuch> happy new chinese year :)
08:01:57 <jose_lausuch> #topic role call
08:02:00 <jose_lausuch> #info Jose Lausuch
08:02:06 <SerenaFeng> #info SerenaFeng
08:02:07 <jose_lausuch> #chair morgan_orange
08:02:07 <collabot> Current chairs: jose_lausuch morgan_orange
08:02:12 <ashishk> #info ashishk
08:02:15 <Cristina__> #info Cristina Pauna (Enea)
08:02:16 <juhak> #info Juha Kosonen
08:02:22 <HelenYao> #info Helen Yao
08:02:24 <LindaWang> #info Linda Wang
08:02:31 <morgan_orange> #info Morgan Richomme
08:03:18 <morgan_orange> #topic Action point follow-up
08:03:28 <jose_lausuch> #topic Action point follow-up
08:03:30 <morgan_orange> #info AP1: morgan_orange indicate keystone v3 status in documentation for Danube
08:03:36 <morgan_orange> #info done, doc updated
08:03:41 <morgan_orange> #info AP2: re-check: LindaWang check path issue with copper on apex virtual pod
08:03:48 <morgan_orange> #info done, copper not supported on fuel or apex yet, confirmed by Bryan, test case disabled in testcases.yaml
08:03:53 <morgan_orange> #info AP3: morgan_orange move api_check/connection_check in healtcheck on Thursday + remove historical healtcheck
08:03:59 <morgan_orange> #info patch in progress https://gerrit.opnfv.org/gerrit/#/c/27923/
08:04:06 <morgan_orange> #info new saps healthcheck needed to cover old healthcheck scope => tests in progress
08:04:11 <morgan_orange> #undo
08:04:11 <collabot> Removing item from minutes: <MeetBot.ircmeeting.items.Info object at 0x1e9e490>
08:04:19 <morgan_orange> #info new SNAPS healthcheck needed to cover old healthcheck scope => tests in progress
08:04:30 <morgan_orange> #info AP4 jose_lausuch check with Steven if that is implemented in SNAPS
08:04:40 <jose_lausuch> #info Done, Steven implemented a healtcheck with DHCP check
08:05:09 <morgan_orange> #link https://gerrit.opnfv.org/gerrit/#/c/28011/
08:05:20 <morgan_orange> #info AP5: morgan_orange follow up on JOID progress
08:05:25 <morgan_orange> #info done, joid non HA back in CI, errors with SNAPS (similar to the ones seen in fuel)
08:05:30 <morgan_orange> #info bug reported to Steve
08:05:35 <morgan_orange> #link https://jira.opnfv.org/browse/FUNCTEST-718
08:05:40 <morgan_orange> #info AP6: juhak investigate why tempest skips 40-50 tests
08:05:44 <juhak> #info tests are skipped due to following services not present:
08:05:51 <juhak> #info sahara, 40 cases
08:05:57 <juhak> #info swift, 14 cases
08:06:05 <juhak> #info and if there is only compute node in setup, also 1 multinode test case is skipped
08:06:09 <jose_lausuch> is sahara part of smoke?
08:06:29 <juhak> hmm... don't know
08:06:37 <jose_lausuch> ok, interesting
08:06:39 <morgan_orange> sounds strange, not really a core feature
08:06:43 <jose_lausuch> thanks for checking that out
08:06:48 <jose_lausuch> it's not
08:06:51 <jose_lausuch> swift could be
08:06:53 <ollivier> Bonjour
08:06:55 <jose_lausuch> but sahara ? :)
08:07:07 <jose_lausuch> ollivier: bonjour bon ami
08:07:31 <jose_lausuch> #info AP 7 morgan_orange: remove non HA to draw daily reporting
08:07:40 <morgan_orange> #info wrong wording, I removed results from virtual POD and kept noHA
08:07:48 <morgan_orange> #info patch submitted to allow filtering on HA/noHA and virtual/Baremetal
08:07:54 <morgan_orange> #link https://gerrit.opnfv.org/gerrit/#/c/27809/
08:07:59 <jose_lausuch> ok!
08:08:07 <ollivier> jose_lausuch: Buenos días
08:08:13 <jose_lausuch> ollivier: :D
08:08:16 <jose_lausuch> #info AP 8 morgan_orange: suggest a cross project review of the documentation to the testing group
08:08:24 <morgan_orange> #info to be done this week, last week meeting was canceled (no quorum)
08:08:38 <jose_lausuch> ok
08:08:47 <jose_lausuch> #action morgan_orange: suggest a cross project review of the documentation to the testing group
08:08:55 <jose_lausuch> #topic Healthcheck tier
08:09:02 <jose_lausuch> let me share some info that you already know
08:09:10 <jose_lausuch> #info we moved api_check and connection_check to Healthcheck Tier.
08:09:31 <jose_lausuch> #link https://gerrit.opnfv.org/gerrit/#/c/27923/
08:09:38 <jose_lausuch> #info Steven created a simple test case in snaps that does the same as our healtcheck
08:09:43 <jose_lausuch> #link https://gerrit.cablelabs.com/#/c/311/
08:09:58 <jose_lausuch> #info it creates a project, user, image, net/subnet, flavor and 1 instance and checks it gets an ip
08:10:06 <jose_lausuch> #info The Healthcheck tier will consist of 3 SNAPS test cases: connection_check, api_check, dhcp/healthcheck
08:10:44 <jose_lausuch> do you all agree with this?
08:10:48 <jose_lausuch> any concerns?
08:10:50 <jose_lausuch> any thoughts?
08:11:02 <jose_lausuch> no more .sh script!
08:11:21 <SerenaFeng> great
08:11:27 <HelenYao> #agreed
08:11:34 <morgan_orange> #agreed
08:11:41 <LindaWang> #agreed
08:11:45 <SerenaFeng> #agreed
08:12:20 <jose_lausuch> #agreed ready to replace .sh healtcheck by the one written by Steven leveraging snaps lib
08:12:29 <morgan_orange> we may note that SNAPS project will be discussed this afternoon during the TSC, if agreed the code will be forked from Cable Labs repo to OPNFV repo
08:12:49 <jose_lausuch> it was discussed also during last week in the tech-discuss meeting
08:12:49 <morgan_orange> it will be easier and more consistent if we need to work directly on SNAPS
08:12:57 <jose_lausuch> yes
08:13:02 <morgan_orange> we can see that currently tehre are still several issues with SNAPS smoke
08:13:08 <jose_lausuch> morgan_orange: will you be there?
08:13:10 <jose_lausuch> at the tsc call?
08:13:11 <morgan_orange> yes
08:13:17 <jose_lausuch> I will be proxing Tim again
08:13:23 <jose_lausuch> so we can support the idea :)
08:13:40 <morgan_orange> yes I assume one of the question that could be raised is the delta with Tempest...
08:13:49 <jose_lausuch> yes, I got that question already
08:14:27 <jose_lausuch> anyway, that's something that cablelabs have to defend
08:15:03 <jose_lausuch> a positive thing is: if the code is in opnfv, changes will be fast
08:15:07 <jose_lausuch> which is true
08:15:23 <HelenYao> one question about snap-oo, will it touch the area out of openstack, e.g. sdn？
08:15:25 <jose_lausuch> getting something done in tempest upstream might be slower (although also important)
08:15:44 <morgan_orange> HelenYao: I think it is one of the possible answer  regarding Tempest
08:15:47 <jose_lausuch> HelenYao: not sure if that is the scope of the project
08:15:49 <morgan_orange> teh scope is not only on OpenStack...
08:15:50 <jose_lausuch> but why not
08:16:22 <jose_lausuch> if the project is in opnfv, we can also push for things easily
08:16:34 <HelenYao> exactly
08:16:48 <jose_lausuch> but first things first
08:16:57 <jose_lausuch> let's see this afternoon
08:17:24 <jose_lausuch> ok, next topic?
08:17:36 <jose_lausuch> #topic Logging level rules / discussion
08:18:05 <jose_lausuch> #info Some tests produce too much output on Jenkins console. Some of the "INFO" messages should be considered "DEBUG"
08:18:10 <jose_lausuch> #info discussion ongoing for SNAPS: #link https://jira.opnfv.org/browse/FUNCTEST-698
08:18:19 <morgan_orange> #info patch done on security_scan
08:18:27 <jose_lausuch> #link https://jira.opnfv.org/browse/SECSCAN-20
08:18:33 <jose_lausuch> #info Luke already worked on that : https://gerrit.opnfv.org/gerrit/#/c/28105/
08:18:35 <jose_lausuch> however!
08:18:42 <jose_lausuch> still, too much useles info. See last run:
08:18:45 <jose_lausuch> https://build.opnfv.org/ci/view/functest/job/functest-apex-apex-daily-master-daily-master/390/consoleFull
08:19:52 <jose_lausuch> mmm but
08:20:08 <jose_lausuch> this run is from Jan 28
08:20:15 <jose_lausuch> the patch was merged last week
08:20:44 <jose_lausuch> #action jose_lausuch check for security scan logs in apex
08:20:53 <SerenaFeng> first of all I think we should also put the log of security_scan to security_scan.log like we did to other feature projects
08:21:01 <jose_lausuch> yes
08:21:07 <SerenaFeng> and aloso SNAPS
08:21:25 <jose_lausuch> yes, we told Steven already
08:21:31 <jose_lausuch> https://jira.opnfv.org/browse/FUNCTEST-698
08:21:35 <morgan_orange> shall we write some rules somewhere?
08:21:52 <jose_lausuch> maybe in wiki
08:22:09 <HelenYao> for the security scan log, it will be fixed by this patch https://gerrit.opnfv.org/gerrit/#/c/28019/
08:22:15 <SerenaFeng> the console log should only indicate the test is success or fail
08:22:20 <HelenYao> it is adopting the featurebase
08:23:03 <jose_lausuch> #action all (specially Luke ): review for sec scan logs https://gerrit.opnfv.org/gerrit/#/c/28019/
08:23:07 <jose_lausuch> SerenaFeng: however
08:23:17 <jose_lausuch> is this removing the code from functest?
08:23:27 <jose_lausuch> ah yes
08:23:30 <jose_lausuch> good
08:23:33 <jose_lausuch> I'll take a look
08:23:46 <SerenaFeng> jose_lausuch, nope
08:23:56 <SerenaFeng> I don't think it will remove the code from functest
08:24:05 <SerenaFeng> just change the way we coding
08:24:11 <jose_lausuch> mmm it removes the opnfv_tests/security_scan :)
08:24:25 <jose_lausuch> and leaves the wrapper caller in features/security_scan
08:24:27 <HelenYao> yes, the code is removed
08:24:36 <jose_lausuch> calling the script in the sec scan repo
08:24:38 <HelenYao> just like what u said in another patch
08:24:40 <jose_lausuch> which is what Luke wanted
08:24:45 <HelenYao> all comments are included
08:24:50 <SerenaFeng> ok
08:25:33 <jose_lausuch> #action jose_lausuch create a wikipage for logging best practicies for Functest
08:26:15 <jose_lausuch> #info basically, output useful information to understand the flow of the test case to INFO
08:26:23 <jose_lausuch> #info the verbose output to DEBUG
08:26:43 <jose_lausuch> #info and everything to a log file that is uploaded to artifacts
08:26:51 <jose_lausuch> I will put some examples
08:26:52 <jose_lausuch> is that ok?
08:27:01 <HelenYao> #agreed
08:27:17 <morgan_orange> #agreed
08:27:22 <jose_lausuch> ok, thanks
08:27:51 <SerenaFeng> #agreed
08:27:57 <jose_lausuch> #topic Functest refactoring
08:29:07 <morgan_orange> MS5 is now over, so in theory, we should only work on bug fixes/troubleshooting...
08:29:10 <jose_lausuch> #info we should cut the refactor activity now
08:29:15 <jose_lausuch> yep
08:29:24 <jose_lausuch> we should focus on fixing things
08:29:39 <jose_lausuch> what about exec_tests.sh ?
08:29:53 <jose_lausuch> what tests are still called by that?
08:30:34 <morgan_orange> hmm still needed for ONOS
08:30:59 <morgan_orange> no more needed for odl
08:31:06 <SerenaFeng> sorry I haven't seen the latest code, is doctor refactored?
08:31:07 <morgan_orange> patch for securoity_scan
08:31:12 <jose_lausuch> https://git.opnfv.org/functest/tree/functest/ci/testcases.yaml
08:31:20 <morgan_orange> SerenaFeng: yes
08:31:40 <morgan_orange> moon I could do it (even if as far as I understood moon will not be in Danube)
08:32:10 <jose_lausuch> #info Needs adaptation to testcasebase/featurebase:  ONOS, ONOS-SFc, Moon
08:32:11 <jose_lausuch> just 3
08:32:11 <morgan_orange> #action morgan_orange moon refactoring (Jira + FeatureBase)
08:32:13 <jose_lausuch> I think we can make it
08:32:18 <SerenaFeng> okey, that's greate I receive a task from doctor people last year, it will base on the refactored version
08:32:28 <jose_lausuch> morgan_orange: if Moon is not in Danube we should remove the tests, right?
08:32:28 <morgan_orange> I need to drop (bring my kid to teh dentist...)
08:32:33 <morgan_orange> do not action myself too much :)
08:32:44 <morgan_orange> jose_lausuch: I will check with moon PTL
08:32:46 <HelenYao> one question - is functest/utils/functest_vacation.py still needed
08:32:53 <HelenYao> if no, I would suggest to remove it
08:33:08 <jose_lausuch> #action morgan_orange check moon status. Might need test refactor if it's in Danube.
08:33:11 <jose_lausuch> just this :)
08:33:31 <SerenaFeng> and also feature/multisite.py
08:33:38 <jose_lausuch> HelenYao: ya, we could keep the script but remove the tests (it's not in testcases.yaml though)
08:33:43 <SerenaFeng> it is useless, multisite test is included in tempest part
08:33:57 <jose_lausuch> SerenaFeng: multisite has the refactor
08:34:04 <jose_lausuch> but with the different Class
08:34:12 <jose_lausuch> https://git.opnfv.org/functest/tree/functest/ci/testcases.yaml#n261
08:34:26 <SerenaFeng> yes, I know, I mean multisite.py is not used
08:35:00 <jose_lausuch> ah
08:35:04 <jose_lausuch> right!
08:35:06 <HelenYao> if script exsits that is not used by others, I would propose to remove it in regard to unit test
08:35:09 <jose_lausuch> klet's remove it
08:35:21 <HelenYao> if the script exists, the unit test coverage rate will be decresased
08:35:21 <jose_lausuch> #action SerenaFeng remove multisite.py
08:35:22 <jose_lausuch> ok?
08:35:26 <ollivier> Please have a look to https://gerrit.opnfv.org/gerrit/#/c/27113/
08:35:32 <SerenaFeng> ok
08:35:54 <HelenYao> how about the decision for functest/utils/functest_vacation.py
08:36:23 <jose_lausuch> HelenYao: you can keep the script, but the test won't be callable from anywhere
08:36:27 <jose_lausuch> ollivier: ok, I will
08:36:45 <jose_lausuch> if we remove exec_tests.sh it shoudl be fine
08:37:10 <HelenYao> if the script is kept, is the unit test case needed for it?
08:37:17 <SerenaFeng> agree to remove it
08:37:35 <SerenaFeng> agree to remove exec_tests.sh
08:37:58 <jose_lausuch> HelenYao: well, it was some joke that Viktor did 2 releases back
08:38:01 <jose_lausuch> it doesn't hurt
08:38:10 <jose_lausuch> but no need for unit tests for such a  thing :)
08:38:20 <HelenYao> see
08:38:22 <SerenaFeng> ollivier: I will review it
08:38:55 <SerenaFeng> I should see I don't like the args part
08:40:06 <jose_lausuch> SerenaFeng: it adds more flexibility when running the tests, why don't you like it?
08:40:13 <jose_lausuch> for example, we could use for tempest
08:40:23 <jose_lausuch> instead of having different classes for Tempest suites
08:40:33 <jose_lausuch> we could have just 1 class and then arguments to run multisite/etc
08:41:30 <SerenaFeng> are there if...else... process in the code?
08:42:05 <SerenaFeng> if so I am afraid it will violate OCP principle
08:42:05 <jose_lausuch> like a normal script having arguments? right ollivier ?
08:42:51 <ollivier> sure. It's fully safe and avoids duplicating code.
08:42:52 <SerenaFeng> the design principle of open-close principle
08:44:30 <jose_lausuch> we can give it a try
08:44:38 <jose_lausuch> but not doing the tempest refactor at this point
08:44:49 <jose_lausuch> you can also leave the comments in the patch
08:44:57 <jose_lausuch> #topic AoB
08:45:03 <jose_lausuch> #info Promise test case disabled until it works https://gerrit.opnfv.org/gerrit/#/c/27999/
08:45:07 <jose_lausuch> #info refactored also in promise repo: https://gerrit.opnfv.org/gerrit/#/c/27553/
08:45:08 <SerenaFeng> it is different tests, one class will be violate another principle, I cannot remember its name now
08:45:37 <jose_lausuch> but it's not compleetelly different tests
08:45:44 <jose_lausuch> it's running tempest with different flavos
08:45:46 <jose_lausuch> flavors
08:46:00 <jose_lausuch> in the end it's just 1 argument that you pass to tempest command
08:46:35 <SerenaFeng> jose_lausuch I agree with you that let's give it a try to odl first
08:46:52 <jose_lausuch> ok
08:46:54 <jose_lausuch> :)
08:47:03 <jose_lausuch> #info Documentation updated. Config Guide, User Guide and Dev Guide. We will have another look before the release but minor updates expected.
08:47:08 <jose_lausuch> anyone wants to share something?
08:47:23 <jose_lausuch> we have time today
08:47:30 <SerenaFeng> about OpenStack import rule
08:47:35 <SerenaFeng> I have a trivial question
08:47:38 <jose_lausuch> aha
08:47:57 <SerenaFeng> should we follow it strictly?
08:48:19 <SerenaFeng> I think part of it is overhead
08:48:19 <ollivier> yes as much as possible
08:48:21 <jose_lausuch> I think ollivier  is pushing for it :)
08:48:32 <SerenaFeng> like import object or module
08:48:33 <ollivier> I know it's hard to only import modules
08:48:47 <jose_lausuch> but one thing to consider is that we don't have those rules in opnfv (we are one of the most tidy projects I would say)
08:48:47 <SerenaFeng> I agree with as much as possible
08:49:13 <ollivier> As much as possible means you should argue if you don't respect it :)
08:49:14 <SerenaFeng> but follow it completely?
08:49:18 <jose_lausuch> ollivier: would you like to propose a patch to be according to that rules?
08:49:28 <jose_lausuch> for example CONST
08:49:38 <jose_lausuch> I think it's pretty conveniento to use
08:49:44 <jose_lausuch> CONST.my_constant
08:49:51 <jose_lausuch> it's simple and short
08:50:02 <HelenYao> the same to testcasebase
08:50:02 <SerenaFeng> yes
08:50:07 <ollivier> Bad exemple. I don't like how we use CONST
08:50:21 <SerenaFeng> the same to testcasebase
08:51:29 <jose_lausuch> why not
08:51:32 <jose_lausuch> what would you change?
08:53:10 <ollivier> No testcasebase follows the rule
08:53:49 <HelenYao> yes, currently, testcasebase follows the rule
08:53:52 <ollivier> (at least before the last review which could have been reverted)
08:55:15 <SerenaFeng> but I think following the rule to process testcasebase or CONST will looks overhead
08:55:31 <SerenaFeng> the question is: does it necessary to process it like this?
08:55:59 <SerenaFeng> I think import testcasebase directly will be simplier
08:56:58 <jose_lausuch> ollivier: any comment on that?
08:57:44 <ollivier> I don't see the point. Both are quite simple
08:58:17 <ollivier> (I think)
08:58:27 <SerenaFeng> the usage
08:58:46 <SerenaFeng> constants.CONST.attribute vs CONST.attribute
08:59:20 <ollivier> and? there is bigger issues in functest :)
08:59:50 <SerenaFeng> no, I think we are working on an agreement
09:00:09 <SerenaFeng> does it necessary to follow OpenStack rule strictly, no exception
09:00:26 <jose_lausuch> I'm not an expert
09:00:27 <SerenaFeng> or we can be a little flexible
09:00:42 <jose_lausuch> but having CONST.attribute instead of constants.CONST simplifies the code a bit
09:00:48 <jose_lausuch> but I'm not the expert :)
09:00:50 <HelenYao> I would say, as we are in the phase of release, how about keeping things as-is (CONST has been spread over the project)
09:00:54 <ollivier> OpenStack doesn't fully follow this rules anyway. But it's fine to follow it if possible
09:01:05 <HelenYao> if we want to stick to the rule, we can fix it in the next phase
09:01:23 <HelenYao> change what it is currently might introduce regression
09:01:34 <SerenaFeng> so no -1 if it is not a bit violation?
09:01:35 <jose_lausuch> HelenYao: I agree with that, I would like to do major changes now
09:01:38 <SerenaFeng> big
09:02:19 <ollivier> I don't think I voted -1 only for that.
09:02:36 <SerenaFeng> okey
09:02:43 <jose_lausuch> ok
09:03:01 <jose_lausuch> I say that we try to implement that in E-river and have the conversation again
09:03:03 <ollivier> If modules are not ordered, I will
09:03:14 <SerenaFeng> agree
09:03:21 <SerenaFeng> that kind of big violation
09:04:10 <HelenYao> agreed with putting the implementation in E-river
09:04:22 <HelenYao> have a happy day, folks~
09:04:40 <SerenaFeng> and another question
09:04:52 <SerenaFeng> should we write unittest from now on?
09:05:29 <SerenaFeng> is the unittest internship almost finished?
09:05:37 <HelenYao> I would put it in E-river, whever a new patch, the unit test should be there
09:05:53 <HelenYao> SerenaFeng: the unittest is not fully finished
09:06:15 <HelenYao> Ashish is mainly working on openstack modules
09:06:26 <ollivier> I think everyone should write unittest... not only Ashish
09:06:28 <HelenYao> I think features will be the next phase
09:06:32 <jose_lausuch> have a good day/evening
09:06:35 <jose_lausuch> we are out of time
09:06:40 <jose_lausuch> #endmeeting