08:01:42 #startmeeting Functest weekly meeting November 7th 2016 08:01:42 Meeting started Tue Nov 8 08:01:42 2016 UTC. The chair is jose_lausuch. Information about MeetBot at http://wiki.debian.org/MeetBot. 08:01:42 Useful Commands: #action #agreed #help #info #idea #link #topic. 08:01:42 The meeting name has been set to 'functest_weekly_meeting_november_7th_2016' 08:01:50 #topic role call 08:02:24 #info Jose Lausuch 08:02:41 #info Helen Yao 08:02:45 #info Morgan Richomme 08:02:57 #info Bertrand Souville 08:03:10 #info Juha Kosonen 08:03:13 #info agenda for today here: https://wiki.opnfv.org/display/functest/Functest+Meeting#FunctestMeeting-08/11(8UTC) 08:03:19 #info previous minutes: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-functest/2016/opnfv-functest.2016-10-18-08.00.html 08:04:47 we have a long agenda today 08:04:47 let' 08:04:50 let's start 08:04:57 #topic OpenStack summit recap 08:05:08 #info we had productive meeting during the summit where many contributors from Functest were present 08:05:23 #info participants: Morgan, Cedric, Juha K., Serena, Helen, Steve and myself 08:05:26 do I miss anyone? 08:05:34 #info SerenaFeng 08:05:39 #info presentation that I used to follow the agenda points https://docs.google.com/presentation/d/15SzJ4NmhrxoVx5LfZ4hPYsDifKoNCLmVA7-oK4abyzw/edit?usp=sharing 08:05:43 #info rohitsakala 08:06:08 #info we covered the main topics that we want to work for Danube: Framework refactor, CI evolution, Docker slicing and integration of SNAPS (which can be renamed). We missed some other smaller tasks, but they are for sure as important as these. 08:06:25 #info there was another important session with Bitergia (Jesus) about integration of our results in their Dashboard. 08:06:39 Morgan, do you want to elaborate that last thing a bit better? :) 08:07:04 yes 08:07:09 but we have a slot for that 08:07:21 #info sync with bitergia, we showed what we did so far 08:07:36 #info we described our different data sets (functest, yardstick) 08:07:42 let' 08:07:48 let's talk about it later then 08:07:49 #info bitergia proposed several scenarios to help us 08:07:53 thanks 08:08:31 as a summary it's perfect, thx 08:08:40 :) 08:08:43 anything else to share about the summit? 08:08:56 I reference the meetup deck in the Danube page https://wiki.opnfv.org/display/functest/Functest+Danube 08:09:25 interesting discussion during the EUAG, seems that it is still not always easy to understand what we are doing 08:09:39 It was nice talking to you F2F at the summit:) 08:09:48 you = everyone 08:09:48 I initiated a discussion at TSC on priority for Danube 08:10:18 #info initiated a discussion at TSC on priority for Danube 08:10:24 HelenYao: it 08:10:35 HelenYao: it's always great to have F2F from time to time 08:10:55 please forgive my dummy finger pressing enter key... 08:10:58 :) 08:11:17 ok 08:11:22 nex topic 08:11:26 we have something to announce 08:11:32 #topic Internships announcements 08:11:53 #info we will have for sure 3 interns in Functest for Danube release. 1 is still pending. 08:12:08 #info paper work is still ongoing, but they will be able to start soon. 08:12:43 #info Ashish Kumar will work on Unit tests with Helen as mentor 08:12:51 #link https://wiki.opnfv.org/display/DEV/Intern+Project%3A+Functest+unit+tests 08:13:04 #info Rohit Sakala will work on the test api project with Serena as mentor 08:13:20 #link https://wiki.opnfv.org/display/DEV/Intern+Project%3A+testapi+evolution 08:13:48 rohitsakala: congrats :) 08:13:55 #info Girish Sukhatankar will work on the security groups test case mentored by David 08:14:01 #link https://wiki.opnfv.org/display/DEV/Intern+Project%3A+Security+groups+test+case+in+Functest 08:14:09 Thank you for the opportunity. 08:14:15 bertys__: on VNF catalog we should also have an intern...we suggest it to a student and got a second candidature... 08:14:32 that is the one pending mentioned by Jose 08:14:39 morgan_orange: thanks 08:14:41 rohitsakala: very welcome ! 08:14:45 something just came up, I have to leave the meeting now, sorry 08:14:46 yep welcome rohitsakala 08:14:55 juhak: ok, no prob 08:15:04 great news indeed 08:15:08 juhak: we will action everything to you :) 08:15:27 :p 08:15:39 welcome rohitsakala 08:15:53 and our third mentor is also on the chan now 08:16:24 morgan_orange: SerenaFeng maybe we could have a chat with rohitsakala about the task when he starts 08:16:57 hangout? 08:17:09 jose_lausuch: SerenaFeng David_Orange as soon as the papers are signed, let's organized a "practice" session 08:17:10 Yes. Sure 08:17:13 yes, why not 08:17:34 how about Ashish Kumar 08:17:35 okey, then I need to fix it as soon as possible :( 08:17:37 morgan_orange: +1 08:17:49 HelenYao: I think he is not here, but the same applies of course 08:18:21 I might need some training about how to be a good mentor 08:18:31 me too 08:18:41 mentor for the first time 08:18:42 HelenYao: I'm mentoring someone and I'm terrible ... 08:18:53 but let's have a hangouts 08:18:57 to talk about that 08:18:58 sorry rohitsakala 08:19:09 I hope I will be a good mentor 08:19:21 jose_lausuch: you might have lessons to tell :-P 08:19:23 I would like to postpone the topic about JIRA assignments sicne I haven't had the time to clean it up 08:19:23 and I will try to be a good mentor 08:19:40 and I will try to be a good intern. 08:19:45 :) 08:19:50 ok, that is very positive! :) 08:20:10 #action Jose: clean up and make clearer our JIRA backlog, and send an email to all. Postpone topic for next week 08:20:27 I like the energy here 08:20:39 bertys__: as soon as I get an answer from the 4th intern, I'll let you know 08:20:53 jose_lausuch: thanks jose_lausuch 08:20:54 otherwise, we can also run another interview if you like 08:21:10 #info 3rd intern is on security group / Mentor: David_Orange 08:21:28 #undo 08:21:50 I infoed already, right? 08:21:56 yes 08:22:06 ok:) 08:22:08 #topic libraries and dependencies installed in Functest image 08:22:12 hot topic now 08:22:22 let me explain the background of this 08:22:25 and let's comment 08:22:32 #info We are having problems in CI due to failed Docker builds 08:22:40 #info I have detected the problem and created this JIRA to keep the track 08:22:48 #info things like pip install requirements try to install the same python modules but with different versions, specially the openstack clients 08:23:04 #info this causes a loss of control over what we are installing 08:23:13 #info for example, the problem of the image build in CI is kingbird requirements "RUN cd ${repos_dir}/kingbird && pip install -e ." 08:23:20 #info we MUST be aware of what we install in the image. 08:23:37 #info we should install the openstack clients that belong to the same openstack release, and not use random versions without any special reason. 08:23:54 #info my proposal: centralize all the requirements installed by any project to a central requirements file and remove all the others (this will also save some lines in the Dockerfile) 08:23:58 thoughts? 08:24:22 I am with the idea of centralization 08:24:47 agree 08:24:54 same for me butĀ could we reach a state where projects requirements would not be compatible 08:24:57 HelenYao: how is OpenStack handling the versions? 08:24:57 openstack runs into the same situation just like us. they have a project : https://github.com/openstack/requirements 08:25:22 aha 08:25:37 I think we could just have a flag to install the os clients 08:25:42 like we do for the repos 08:26:00 but the versions of the python clients are not all the same 08:26:06 can some one dig into that? 08:26:08 for example 08:26:15 installing mitaka client versions 08:26:24 something that we can update later on 08:26:24 or maybe we can install the requirements when the feature project is executed 08:26:30 we might need to do a general scanning for all the projects 08:26:31 for newton or ocata for example 08:26:34 just changing a flag 08:26:47 i can work on this 08:26:53 I detected some lines in the 08:26:58 Dockerfile doing that 08:27:07 tempest, rally, kingbird 08:27:15 sfc (but I already removed that) 08:27:24 ok, thanks 08:27:29 i saw we hardcoded the project tag info in dockerfile, how is that value decided? 08:27:53 #action HelenYao check what are the versions of the OS python clients related to a stable version of OS (starting with Mitaka) 08:28:18 HelenYao: it's not decided, we lost the control of that 08:28:28 I think we started having the right versions in Brahmaputra 08:28:37 but now it's a mess of versions 08:29:09 we have to restrict functest to a stable 3rd project to avoid their messy change 08:29:23 #action HelenYao detect with lines in the Dockerfile are altering the OS python clients (i.e. tempest, rally, kingbird, ) 08:29:31 yes 08:29:45 the kingbird version change was introduced in Sep and our build failed after then 08:29:47 and also, for some other common libraries, like scp for example 08:29:50 could it be manage through the abstraction created by Cedric trhough a prepare_env() or similar 08:29:58 sometimes we install scp with a fix version and other projects install it and upgrade it 08:30:07 then we would check the project needs and centralize the requirement 08:30:08 it would be nice to detect that as well, but it's not so critical 08:30:49 morgan_orange: yes, if needed, but if we can have something common for everyone, it would be perfect, no extra effort, we just need to agree on the versions 08:31:04 and I think we should use the clients that correspond to the System Under Test 08:31:12 if we are testing mitaka, then use the mitaka clients 08:31:13 and so on 08:31:42 jose_lausuch: makes sense 08:32:02 taking the kingbird for example, it requires neutron > 5 and most of the projects only need 4. how can we pursude kingbird? 08:32:14 so first thing => audit of the existing and centralizsation somewhere with lib aligned with the target version 08:32:23 there is a chance that, for mikata, version 4 is enought 08:32:54 HelenYao: ok, feel free to propose a patch updating the versions of our requirements file 08:33:26 we could create a requirements_os.pip or something like that 08:33:30 only for the OS clients 08:34:17 yeah, os clients are heavily used among projects 08:34:53 good 08:34:54 most of the version problems seem like client-stuff 08:35:09 yes, at least in the build process 08:35:20 we need to have more control about what pip install does 08:35:25 good discussion 08:35:25 which brings us to the next topic 08:35:33 #topic Docker follow-up discussion and possible decision 08:35:53 #info I created this patch to show how we could slice the image in the different categories https://gerrit.opnfv.org/gerrit/#/c/23729/ 08:36:00 please all have a look and comment 08:36:15 I posted the sizes of the image building in the associated JIRA 08:36:54 sorry, not in JIRA, in the wiki 08:36:56 #link https://wiki.opnfv.org/display/functest/Docker+images+slicing 08:37:40 https://thepasteb.in/p/Q1hBnqPB4RjS8 08:37:55 one trivial question about dockerfile, for ruby, we hardcoded a line with version 1.9.3 and it warned that the version is no longer maintained 08:38:08 HelenYao: for that, we need to ask Valentin 08:38:15 that is used for vIMS I think 08:38:20 morgan can you contact him? 08:39:00 #action morgan_orange contact valentin regarding hardcoded ruby version that triggers warning 08:39:18 I think it is for the test in vIMS 08:39:41 we should remove the hardcoded version anyway 08:39:45 and adapt the script after 08:39:57 that would be great 08:40:18 at the moment we run vIMS only manually (it is planned for weekly jobs that are currently never ru) 08:40:22 jose_lausuch: did you push the image w/o kingbird to functest:latest? 08:40:34 so we can remove this I think 08:40:35 HelenYao: yes, I did 08:40:59 i tried with the latest and the neutron problem is still there 08:41:10 the installer build is still failing 08:41:11 morgan_orange: is valentin still allowed to commit ? :) 08:41:36 HelenYao: really? 08:41:37 mmm 08:41:44 can you check what keystone version is installed? 08:41:49 anyway, we can discuss it later 08:42:15 #action HelenYao SerenaFeng morgan_orange and all check docker slicing proposal and provide comments 08:42:25 need to move to next topics unfortunatelly 08:42:41 or maybe we can spend 5 more minutes 08:43:10 vijayendra radhakrishna proposed functest: Restructure SFC-ODL test cases https://gerrit.opnfv.org/gerrit/23969 08:43:24 #action complete the slide with the "+" and the "-" about the docker slicing impacts 08:43:35 #undo 08:43:35 Removing item from minutes: 08:43:44 #action jose_lausuch complete the slide with the "+" and the "-" about the docker slicing impacts 08:44:38 HelenYao: you were working on a script to update the docker daemon, right? 08:44:52 yeah. the script is ready 08:45:15 is it in gerrit? 08:45:30 no. i can commit it 08:45:48 where/which folder shall i put it 08:45:59 yes, please, although we still haven't a decission, it's ok to have it in gerrit, like mine with the Dockerfiles 08:46:04 in /docker 08:46:17 sure. i will commit it after the meeting 08:46:33 ok, thanks 08:46:44 i can test it on my local env as well 08:47:04 ok 08:47:09 awesome 08:47:11 #topic Unit tests (repo structure/logger issues) 08:47:15 morgan? 08:47:21 yep 08:47:21 #chair morgan_orange 08:47:21 Current chairs: jose_lausuch morgan_orange 08:47:32 #info introduction of unit tests done in CI 08:47:52 #link https://build.opnfv.org/ci/view/functest/job/functest-verify-master/ 08:48:03 the coverage report seems good 08:48:12 #info only ODL test today but system working locally and in Jenkins including coverage 08:48:42 #info however coverage is limited to the first tests we have (we clearly should extend) and I had to twist the config to make it work 08:48:46 #info 2 main issues 08:49:03 #info only applied for master branch 08:49:09 #info 1 the repo structure: all our code is at the root level => issue with modules 08:49:35 #info we need to create a functest subdirectory in our repo to be sure that we will always have the same structure 08:49:44 today to make it work I need to cd .. 08:50:18 what is the advantage of creating the functest dir then? 08:50:26 please have a look at (after meeting) https://build.opnfv.org/ci/view/functest/job/functest-docker-build-push-master/lastFailedBuild/console 08:50:28 #info 2 in some of our util class, we use the logger that logs in harcoded file 08:51:22 jose_lausuch: if we create a functest dir in our repo we can be sure we will have always the same tree 08:51:37 at the moment on verify we clone the repo as functest-verify-master 08:51:49 with the __init__.py at the root of the repo 08:51:56 we do not have functest.core ... 08:52:05 can't we clone it just as "functest" ? does it make sense? 08:52:29 no because you can clone as you want..we cannot force people to always clone as functest 08:52:38 if they clone differently unit tests will not work 08:52:39 aha ok 08:52:46 if we use functest subdir, it will always work 08:52:57 well, we though of that only working for our container 08:53:03 if we do it outside the container it doesnt work 08:53:07 yes 08:53:14 About the advantage, by creating a functest folder inside and make it as a module, it will not run into the case morgan mentioned. also, by looking at other projects, opnfv and openstack, they are following the pattern we proposed 08:53:16 but functest was though at the beginning to work only inside the container 08:53:17 and CI gating is done out of our containeur 08:53:46 jose_lausuch: also the unit tests? 08:53:48 and then our python path variable will point to the functest root dir right? 08:53:55 jose_lausuch: yes 08:53:55 but unittest will not run under container 08:54:04 no, now it's a new thing 08:54:14 so ok 08:54:18 we need to do it 08:54:21 to be safe 08:54:22 unittest shall be runnable anywhere.... 08:54:28 +1 08:54:36 and we will just follow OpenStack and other project pattern 08:54:54 #info decission on creating functest root dir and move all the tests and tools inside 08:55:04 where do we put unit tests? 08:55:14 does it make sense to put it also in /functest? 08:55:18 or separate root dir? 08:55:24 as you proposed 08:55:24 yes (last discussion initiated by helen) 08:55:29 functest/tests/unit 08:55:32 how about funtest -> tests -> units 08:55:49 just a question on shall it be functest/tests/unit-tests, functest/unit-tests, functest/tests/units 08:55:53 units -> unit 08:55:53 we already have a testcases dir 08:56:13 I would like to avoid confusion 08:56:20 I think it's ok 08:56:21 i think the name of testcases is a bit confusing 08:56:36 all OpenStack project put unit tests under xxx/tests/unit 08:57:00 it seems like tests/unit is defined to be unit test 08:57:01 ok, if we say functest/tests/unit, where do we put the testcases? 08:57:10 yes but OpenStack projects are not test projects as we are... 08:57:16 Rally follow the same rule? 08:57:17 yeah, agree with Serena. I am going for functest/tests/unit 08:57:41 what about this 08:57:43 to functest, testcases is a module 08:58:00 not a tests 08:58:11 we move all the testcases into functest/tests/ 08:58:39 just check rally follow the same design pattern tests/ci tests/functional tests/unit 08:58:42 the tests fold is meant to include the script to verify the project itself. we might need to work out a new name for 'testcases' as it is about the project biz 08:59:06 morgan_orange: +1 08:59:36 no for move all testcases into functest/tests 08:59:53 no + 1 09:00:08 SerenaFeng: then, where do we put them? 09:00:16 no + 1 = -1 ? :) 09:00:20 thinking 09:00:30 https://github.com/openstack/rally 09:00:31 my bad :-P 09:00:44 https://github.com/openstack/rally/tree/master/tests 09:01:11 https://github.com/openstack/tempest 09:01:23 no tests folder in tempest :) 09:01:36 oops 09:01:39 no 09:01:47 https://github.com/openstack/tempest/tree/master/tempest 09:01:50 yes, there is https://github.com/openstack/tempest/tree/master/tempest/tests 09:02:18 Merged functest: Restructure SFC-ODL test cases https://gerrit.opnfv.org/gerrit/23969 09:02:21 they have tests folder 09:02:29 yes my bad :) 09:02:38 then? 09:02:45 what do we agree on? :) 09:03:12 I would say follow the OpenStack way create tests/unit 09:03:14 to me, the 'testcases' is similar to 'services' in rally 09:03:17 push unit tests here 09:03:25 and think if we should rename/move testcases 09:03:38 agree 09:03:38 but I am in line with Helen 09:03:43 ok 09:03:49 #info agree to rename testcases 09:03:51 rename to what 09:03:54 services? 09:03:55 so we can agree on just creating tests/unit 09:04:00 don't know how to express my thinking in English 09:04:06 rename would be great 09:04:13 we agree on creating /functest/ and /functest/tests/unit 09:04:15 +1 on tests/unit 09:04:55 /tests/unit or /functest/tests/unit ? 09:05:03 is "tests" root? 09:05:04 ollivier: I think you missed the Daylight saving time change :) 09:05:06 functest/tests/unit 09:05:13 yes 09:05:26 ok 09:05:29 put tests under subdir functest 09:05:42 who is doing the change.. 09:05:55 morgan_orange: oops... sorry 09:05:56 functest/tests/unit. i was just a bit lazy to input the full path:( 09:05:57 #action morgan_orange propose a patch with the new dir structure as initiated by email 1) Create /functest 2) create /functest/tests/unit 09:06:09 jose_lausuch: ok 09:06:33 i can work on the change 09:06:44 ok 09:06:45 tests/unit is redundant if we don't add functional tests, isn't it? 09:06:53 we are over time 09:06:57 ollivier: for the moment yes 09:07:04 but we just follow OpenStack pattern 09:07:15 morgan's idea and mine was to have /functest/tests/unit /functest/tests/functional/ 09:07:17 and may add additional tests later... 09:07:20 we have to leave the door open for other level of tests 09:07:43 there are functional tests in OpenStack... I don't think we will add them. But we can follow these guidelines. 09:08:18 ollivier: yep there was a discussion on are our testcases functional tests or services if we consider OpenStack naming 09:08:26 so far for me it is more services 09:08:35 we do not have functional tests on the framework itself 09:08:41 but the name "services" is more confusing that "testcases" ? 09:08:55 no rush for renaming... 09:09:07 yeah, no rush for renaming 09:09:09 #agree 09:09:13 then we will have 2 dirs "tests" and "testscases" :) 09:09:17 agree, we can come up with a better name later 09:09:39 naming is always a headache 09:09:48 #agree 09:09:50 maybe testcases should be renamed instead. 09:09:59 ollivier: yes, we were saying that 09:10:09 1 porposal is renamed it to "services" 09:10:22 a good name might occur while taking a bath:-P 09:10:44 HelenYao: that's a great idea... 09:10:59 #action jose_lausuch follow up discussion on dir structure and naming 09:11:21 ok 09:11:27 we have 1 topic left, but I think there is no time 09:11:31 and morgan_orange already said some words 09:11:35 do you want to add something else? 09:11:50 i may work on the structure over the time before next meeting 09:12:21 HelenYao: perfect, propose a patch and we can review it and follow up the discussion there 09:12:22 thanks 09:12:29 sure 09:13:29 #action jose_lausuch move bitergia discussion for next weekly 09:13:33 #endmeeting