08:01:12 <jose_lausuch> #startmeeting Functest weekly meeting 14 Nov. 2017
08:01:12 <collabot> Meeting started Tue Nov 14 08:01:12 2017 UTC.  The chair is jose_lausuch. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:01:12 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:01:12 <collabot> The meeting name has been set to 'functest_weekly_meeting_14_nov__2017'
08:01:32 <jose_lausuch> #info Agenda https://wiki.opnfv.org/display/functest/Functest+5.+Meeting#Functest5.Meeting-14/11(8UTC)
08:01:35 <jose_lausuch> #topic role call
08:01:39 <jose_lausuch> #info Jose Lausuch
08:01:51 <LindaWang> #info Linda Wang
08:02:00 <viktor_t> #info Viktor Tikkanen
08:03:10 <ollivier> #info Cédric
08:03:37 <jose_lausuch> guys, did you cast your votes?
08:04:07 <viktor_t> yep
08:05:10 <ollivier> Thank you!
08:05:50 <jose_lausuch> ok, just seen it
08:05:53 <jose_lausuch> LindaWang ?
08:06:24 <LindaWang> yes
08:06:29 <LindaWang> i have just voted
08:06:37 <jose_lausuch> ok, thank you
08:06:55 <jose_lausuch> #topic action points follow up
08:07:19 <jose_lausuch> #info AP 	1.	depo_ look into the docker automated builds and propose a fix
08:07:26 <jose_lausuch> depo_: ping
08:08:17 <ollivier> please see https://gerrit.opnfv.org/gerrit/#/c/47147/
08:09:04 <jose_lausuch> #info last fix provided https://gerrit.opnfv.org/gerrit/#/c/47147/
08:09:04 <ollivier> The jobs have been rewritten to conform with Travis config files. Mainly we switch the entry point to build.sh
08:09:31 <jose_lausuch> our build.sh in functest
08:09:48 <ollivier> The key is related to the common Docker helper which is not good for Functest (targets simply differ).
08:10:29 <jose_lausuch> ok
08:10:41 <jose_lausuch> shall we have the code out of the job?
08:10:48 <jose_lausuch> in a separate file?
08:10:53 <ollivier> I must implement the container garbage collecter in build.sh as well as releng uses the same VM for building all containers.
08:11:35 <jose_lausuch> let me review it after the meeting
08:11:40 <ollivier> Travis-ci spawns a new VM per job which avoids this issue over the time (it also allows installing requirements by jobs)
08:11:52 <jose_lausuch> ah
08:11:57 <jose_lausuch> our build servers arestatic
08:12:08 <jose_lausuch> they are already configured virtual servers
08:12:32 <jose_lausuch> #info AP ollivier jose_lausuch request a new repo to LF for xtesting
08:12:42 <ollivier> The current garbage collector is false as it removes first functor-core which is key for the other containers.
08:13:06 <jose_lausuch> #info not done yet. Wiki to be completed
08:13:13 <ollivier> yes but I think we must first finish jjob simply because we are publishing arm64 image for latest.
08:13:24 <jose_lausuch> yes
08:13:54 <ollivier> xtesting can wait a little bit.
08:14:00 <jose_lausuch> sure
08:14:31 <jose_lausuch> #topic Functest project organization
08:14:32 <ollivier> we are already applying the pre-requisite work in Functest (see Rally tempest moved out of functest-core)
08:14:52 <jose_lausuch> yes, that's very good
08:15:03 <jose_lausuch> next thing would be to remove prepare env I guess
08:15:22 <jose_lausuch> let's discuss that later
08:15:24 <LindaWang> yes, i am gonna work on that
08:15:31 <ollivier> Please see https://gerrit.opnfv.org/gerrit/#/c/47143/ too .
08:15:53 <jose_lausuch> let me congratulate you
08:16:09 <jose_lausuch> we are missing some votes but I think there is majority of +1
08:16:20 <jose_lausuch> so, congratulate the new PTL Cedric!
08:16:24 <LindaWang> Congrats, Cedric!
08:16:28 <jose_lausuch> :)
08:16:59 <juhak> Congratulations Cedric
08:17:08 <ollivier> Thank you much everyone. I will do my best to apply my best as you (Morgan and Jose) have done.
08:17:34 <ollivier> I will do my best as you have done.
08:17:46 <jose_lausuch> you will do great
08:17:55 <jose_lausuch> you can always ask any question to us, we will help
08:18:32 <ollivier> Yes I know :)
08:18:47 <jose_lausuch> #info After a formal vote on the mailing list, Cedric Ollivier is the new PTL as of today
08:19:17 <jose_lausuch> do you have any idea about the project organization (admin stuff) ?
08:19:35 <jose_lausuch> for example, how do we continue with JIRA
08:20:07 <jose_lausuch> we have some planning in the wiki, but we haven't set priorities as such, although they are almost clear
08:21:11 <ollivier> Yes. I will focus on that right after the jjobs. I think everything has already been written by email. I will complete the wiki and create the JIRA topics.
08:21:27 <jose_lausuch> ok
08:21:32 <jose_lausuch> I created already some epics
08:21:47 <ollivier> Yes. I must complete them. Thank you.
08:21:56 <OPNFV-Gerrit-Bot> Merged functest: Move rally and tempest in functest-restapi  https://gerrit.opnfv.org/gerrit/47143
08:21:59 <jose_lausuch> do you agree with having epics for the "big things" and creating tasks for the smaller things?
08:22:45 <LindaWang> do we really need a story in Jira?
08:23:13 <jose_lausuch> I normally create epics and inside those epics, tasks
08:23:29 <jose_lausuch> I don't know the difference of story and task....
08:23:32 <LindaWang> you created two stories
08:23:38 <ollivier> I will conform with the current process. I think gerrit topics are the key points. But if I'm fine to duplicate them in Jira if asked.
08:24:37 <jose_lausuch> ah right, maybe I confused them with task… they can be changed to task…
08:24:41 <ollivier> But please be free to add your ideas if they aren't listed in my previous email.
08:24:45 <jose_lausuch> to me there is no much difference
08:24:57 <jose_lausuch> yes
08:25:10 <jose_lausuch> I will try to add more things and you can then correct or add more
08:25:16 <jose_lausuch> but it's not urgent
08:26:14 <ollivier> Yes. Let's finish the jjobs first because we are breaking gating.
08:26:40 <jose_lausuch> #topic Docker builds follow up
08:26:52 <jose_lausuch> #info The jobs have been rewritten to conform with Travis config files. Mainly we switch the entry point to http://build.sh/
08:27:11 <jose_lausuch> #info The key is related to the common Docker helper which is not good for Functest (targets simply differ).
08:27:17 <jose_lausuch> just gathering what you said before
08:27:32 <jose_lausuch> what else is needed from this task?
08:28:13 <ollivier> yes. A second step will be to clean it a little bit (useless variables) but it's not as urgent. I think we should merge it.
08:28:36 <jose_lausuch> #info add cleaning at the beginning of the script
08:28:47 <ollivier> Then we must check that manifest-tools are installed in VM. Of course tests are needed if anyone can help.
08:28:50 <jose_lausuch> well, the cleaning mechanism we have today is not perfect either
08:28:59 <jose_lausuch> I have to login to the server some times to do a manual clean
08:29:10 <jose_lausuch> I want to automate that, as it's just running 2 lines
08:29:44 <jose_lausuch> ollivier: I can try to install that if it's not
08:29:51 <jose_lausuch> what libraries are needed?
08:29:53 <ollivier> ok. The helper considers only one VM per project. That's false regarding our needs
08:30:55 <ollivier> https://git.opnfv.org/functest/tree/.travis.yml#n9
08:31:00 <ollivier> https://git.opnfv.org/functest/tree/.travis.yml#n10
08:31:27 <jose_lausuch> I will install it
08:31:28 <ollivier> It runs a new container. And then you can simply check by calling sudo manifest-tool -h
08:31:33 <jose_lausuch> on ericsson build 3 and 4
08:31:50 <ollivier> Maybe Fatih did that yesterday (I sent him the links). Thank you.
08:32:51 <jose_lausuch> mmmm ok, I'' check with him
08:33:19 <ollivier> Regaring cleaning we must updaet build.sh to clean the previous build. The key point is to remove before deleting functest-core.
08:33:31 <ollivier> to remove sons...
08:33:46 <jose_lausuch> we have to keep our own clean
08:34:16 <ollivier> Yes simply because of Docker slicing.
08:34:19 <jose_lausuch> that's right
08:34:23 <jose_lausuch> dependencies between images
08:34:49 <jose_lausuch> it's not difficult to implement that cleaning
08:35:17 <jose_lausuch> #info cleaning mechanism is to be added to our build.sh to remove childs before functest-core
08:35:28 <LindaWang> Do you think functest-smoke should be based on functest-components?
08:35:35 <ollivier> No the main question is before building or after.
08:35:37 <LindaWang> i saw your comments yesterday
08:36:06 <jose_lausuch> if we do it before, we make sure we have a clean environment before we build
08:36:13 <ollivier> Yes. It's the best approach but we have to first fix the jenkins jobs before adding other dependencies in containers
08:36:38 <LindaWang> will it be a bit complicated for the jenkins jobs and cleaning?
08:37:22 <ollivier> then euphrates and master jobs will strongly differ but from a docker pov,  functest-smoke should be based on functest-components
08:37:59 <jose_lausuch> yes, that will be a problem….
08:38:14 <LindaWang> but it is not urgent i guess. maybe we could consider that later when the jenkins is ready
08:38:18 <ollivier> Not so difficult. The multijobs will simply differ a little bit and can be aggregated as it is from the time being
08:38:21 <jose_lausuch> 2 options, either we wait until euphrates 2.0 is out and do that
08:38:38 <jose_lausuch> or we add a if branch == master/euphrates. in the jjob
08:39:00 <ollivier> I think we should first finish the current jobs for Euphrates 2.0. Fixing jjobs, finish snaps staff and integrate vEPC.
08:39:07 <jose_lausuch> yes
08:39:15 <jose_lausuch> agree, that is not urgent
08:39:20 <LindaWang> we should also remove the functest related docker build job defined in opnfv-docker-arm.yml
08:39:25 <jose_lausuch> that's. why it's important to set priorities
08:39:40 <ollivier> Regarding F we are already closed  to the minimal staff (switching to Pike).
08:39:44 <jose_lausuch> LindaWang:  shall I action you?
08:40:10 <LindaWang> i must make sure the jenkins is ready first
08:40:13 <LindaWang> sure jose_lausuch
08:40:19 <ollivier> Yes. I forgot to add it in my review.
08:40:28 <jose_lausuch> #action LindaWang remove functest related docker build job defined in opnfv-docker-arm.yml
08:40:31 <jose_lausuch> thanks
08:40:31 <ollivier> I think we can remove it as we removed the Dockerfile.
08:40:37 <LindaWang> ollivier: OK
08:40:37 <jose_lausuch> yes
08:41:22 <LindaWang> And more, the set-functst-env.sh should also be updated for arm
08:41:31 <jose_lausuch> ah right
08:41:41 <jose_lausuch> as they are using the old jjob
08:41:47 <LindaWang> set-functst-env.sh is only used by arm currently, right?
08:42:27 <ollivier> And there is an opening question about check_deployment (prepare_env.py )as well.
08:42:58 <jose_lausuch> LindaWang: I will help you with that, as I know how the functest job are structured
08:43:03 <jose_lausuch> yes
08:43:06 <jose_lausuch> I wanted to discuss that as well
08:43:18 <jose_lausuch> check_deployment.py is actually useful some times
08:43:27 <LindaWang> In jenkins, if the prepare_env is ready, shall we remove the step of prepare env in the following containers.
08:43:28 <jose_lausuch> to avoid running tests when there is no connection or something
08:43:48 <ollivier> jose_lausuch:  could you please merge https://gerrit.opnfv.org/gerrit/#/c/47147/ ? Then everyone can test the results.
08:43:58 <jose_lausuch> btw, I have already installed the manifest-tool on build server 3 and 4
08:44:17 <LindaWang> or we will remove prepare env and only keep check deployment?
08:44:28 <ollivier> thank you. Then we can safely merge the change about releng as it can't break more ;)
08:44:31 <jose_lausuch> merged
08:44:53 <ollivier> thank you. Let's see if we can build the whole chain (it works locally)
08:44:54 <jose_lausuch> we can keep check deployment in healtcheck
08:45:01 <jose_lausuch> as a command before running the tests...
08:45:02 <jose_lausuch> maybe
08:45:37 <LindaWang> jose_lausuch:  only kept in healthcheck container?
08:45:47 <jose_lausuch> today, we run checkdeployment in every container because it's in prepare env
08:45:52 <ollivier> In fact the purpose is to remove the file. If several steps are useful we must simply move it to TestCase classes.
08:46:06 <jose_lausuch> if we remove prepare env, I would at least keep it at the beginning
08:46:19 <jose_lausuch> or it can be called directly from the jenkins job
08:46:46 <ollivier> If they are common operations for all testcases they could be ran in the mother class.
08:46:50 <jose_lausuch> but it will avoid wasting time running test cases if there is something wrong
08:47:14 <jose_lausuch> it's basically checking that the public endpoint is reachable and some other minor things
08:47:23 <ollivier> Yes but why not creating a initial step in our framework.
08:47:26 <jose_lausuch> doing that only 1 time at the beginning is enough
08:47:34 <jose_lausuch> ok
08:47:40 <jose_lausuch> but it's openstack related
08:47:42 <LindaWang> move it to TestCase classes? as Cedric suggested?
08:48:06 <jose_lausuch> it could be the very first test… maybe
08:48:22 <ollivier> In the context of xtesting, we must first create new classes (eg OPNFVTestCase on top of Testcase)
08:48:25 <jose_lausuch> ollivier: are you suggesting it as a new test?
08:48:41 <jose_lausuch> before connection_check?
08:48:55 <jose_lausuch> or maybe connection_check already does that… I'm not sure
08:49:29 <jose_lausuch> our check is simpler and is more a gate to run tests
08:49:39 <jose_lausuch> I would suggest to keep it
08:49:40 <LindaWang> connection_check does basic client connection
08:50:03 <ollivier> I think we could create the basic framework in xtesting (no OPNFV/OpenStack links). Then implement new classes in OPNFV on top of them. I think we could move several operations located in prepare_env directly in a new Functest mother class.
08:50:04 <LindaWang> check deployment only checks the nova neurton and glance endpoint?
08:50:23 <jose_lausuch> LindaWang: yes
08:50:40 <jose_lausuch> ollivier: yes, but in order to remove it already from prepare_env...
08:51:02 <jose_lausuch> that's the main thing in prepare_env, now that rally installation is out of it
08:52:19 <LindaWang> The issue is snaps could be kept in xtesting?
08:52:24 <ollivier> We can create a mail thread to discuss on that. I think loading env vars for instance could be done in a mother class.
08:52:40 <jose_lausuch> ok
08:52:42 <ollivier> I don't think so as snaps is OpenStack related.
08:52:56 <jose_lausuch> LindaWang: no, xtesting is VIM agnostic
08:52:58 <ollivier> Functest could depend on snaps
08:53:05 <jose_lausuch> that's the main idea of xtesting
08:53:46 <LindaWang> then we'll remove releng/opnfv module as well.
08:53:54 <ollivier> If snaps is well covered by functional tests as it seems, Functest master can rely on snaps master.
08:54:10 <LindaWang> now releng/opnfv is only used by sfc running on XCI?
08:54:10 <jose_lausuch> LindaWang: would you like to share some words about the status of k8 testing?
08:54:27 <jose_lausuch> LindaWang: yes, we can't remove it,they needed
08:54:43 <LindaWang> Konrad is working on a presentation of k8s test cases in test-infra
08:54:49 <ollivier> But we have to be careful that the subset we are defining won't be changed over the release (eg a new tests out of OPNFV requiremetns)
08:54:51 <jose_lausuch> #AoB
08:54:59 <jose_lausuch> #info Konrad is working on a presentation of k8s test cases in test-infra
08:55:11 <LindaWang> when it comes to snaps, how  often shall the subset of snaps testcases in functest be updated?
08:55:12 <jose_lausuch> ollivier: what do you mean?
08:56:00 <LindaWang> do you think we should create a new test case to run the experimental tests newly developed in snaps?
08:56:30 <ollivier> jose_lausuch: snaps could introduce a new test in basic services (Neutron/Nova/Heat) which break our gating. At least now we don't test any magnum operations.
08:56:41 <jose_lausuch> ah ok
08:56:49 <jose_lausuch> well, we can frequently talk to steven
08:56:55 <jose_lausuch> if we have the control of which tests are run
08:57:00 <jose_lausuch> we have the decission
08:57:10 <ollivier> yes that's the good point of snaps. It's easy to discuss with Steven.
08:57:23 <jose_lausuch> I wouldn't include experimental tests at least in smoke
08:58:22 <jose_lausuch> I have something else to share
08:58:47 <jose_lausuch> we will have new ODL test cases
08:58:59 <LindaWang> i am afraid if we do not update the testcases over a release, we will run into some trouble debugging new issues after a new release
08:59:48 <ollivier> LindaWang: Ofc we have to update the testcase as we are doing for rally.
09:00:09 <LindaWang> ollivier: ok
09:00:47 <jose_lausuch> let's discuss it with Steven, I think his point of view is also important
09:00:50 <ollivier> We should update the list of tests, the versions... The key point about snaps is to integrate a part of it. Snaps goes fast and is not limited to OPNFV context.
09:00:57 <jose_lausuch> I will send an email about the new ODL tests
09:01:14 <jose_lausuch> I have to go to another meeting
09:01:15 <ollivier> Yes please. Does it concern new robot files$?
09:01:33 <ollivier> OK. Thank you for the meeting.
09:01:37 <jose_lausuch> ollivier: that's the question and the discussion
09:01:44 <jose_lausuch> #endmeeting