08:00:14 <morgan_orange> #startmeeting Functest weekly meeting February 9th
08:00:15 <collabot> Meeting started Tue Feb  9 08:00:14 2016 UTC.  The chair is morgan_orange. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:00:15 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:00:15 <collabot> The meeting name has been set to 'functest_weekly_meeting_february_9th'
08:00:21 <morgan_orange> #info Morgan Richomme
08:00:23 <viktor_nokia> #info Viktor Tikkanen
08:00:50 <juhak> #info Juha Kosonen
08:01:03 <morgan_orange> hi viktor_nokia sorry I did not see your patch yesterday evening, I merged this morning and run Tempest on Orange POD 2 to check
08:01:18 <viktor_nokia> OK!
08:01:29 <morgan_orange> https://build.opnfv.org/ci/view/functest/job/functest-joid-orange-pod2-daily-master/65/console
08:01:37 <morgan_orange> #topic action points follow-up
08:01:59 <jose_lausuch> #info Jose Lausuch
08:02:03 <morgan_orange> #info AP1 juhak test performed on Cinder on Orange POD 2 done
08:02:19 <morgan_orange> #info AP2 morgan_orange tempest.conf found
08:02:42 <morgan_orange> #info AP3 morgan_orange check delete scnearion in ODL not done, we will discuss ODL tests in next section
08:02:54 <morgan_orange> #info AP3 juhak troubleshooting on heat done
08:03:16 <morgan_orange> #info AP5 jose_lausuch push vPing into DB in case of Timeout done
08:03:37 <morgan_orange> #info AP6 morgan_orange propose test strategy done (shared on the Brahmaputra etherpad)
08:03:55 <morgan_orange> #info AP7 leo_wang access to compass lab for Valentin not done
08:04:12 <morgan_orange> #info AP7 complete etherpad Functest for Release C initiated
08:04:25 <morgan_orange> #topic  Release B
08:04:38 <morgan_orange> #info discussions ongoing on the date for release
08:04:59 <morgan_orange> #info late decission to move from Li to Be but a priori only progress expected..
08:05:02 <jose_lausuch> will it be decided today?
08:05:07 <morgan_orange> probably
08:05:22 <morgan_orange> #info a date will be probably decided during TSC meeting today
08:05:39 <jose_lausuch> ok
08:05:51 <morgan_orange> #info concerning the status of Functest for Brahmaputra 4 new pages have been created on the alternative dashboard
08:06:00 <morgan_orange> #link http://testresults.opnfv.org/proto/brahma/index-apex.html
08:06:05 <morgan_orange> #link http://testresults.opnfv.org/proto/brahma/index-compass.html
08:06:11 <morgan_orange> #link http://testresults.opnfv.org/proto/brahma/index-fuel.html
08:06:17 <morgan_orange> #link http://testresults.opnfv.org/proto/brahma/index-joid.html
08:06:33 <morgan_orange> #info Doctor enabled yesterday
08:06:55 <morgan_orange> #info only OVNO not done as planned but no ocl deployement available with any installer
08:07:09 <morgan_orange> #info regarding the status Functest versus installer for the release
08:07:30 <morgan_orange> #info Compass => green light ready for release (except ocl scenario but no ocl deployement)
08:08:09 <morgan_orange> #info fuel => almost green (vPing issue on odl_l3 scenario, need to boost Tempest from ~80% to 90 to reach criteria)
08:08:48 <jose_lausuch> but viktor is doing a good job detecting the cause of the failures :)
08:08:57 <viktor_nokia> waiting for available tempest.los's...
08:09:02 <viktor_nokia> log's
08:09:06 <jose_lausuch> btw, when we don't have shared store...
08:09:06 <morgan_orange> #info apex => not far from green (more issues on vPing (onos and odl) same story with Tempest to be boosted)
08:09:14 <jose_lausuch> what shall we do with live migr?
08:09:38 <morgan_orange> #info joid => main concern when running on target POD lots of errors Tempest, promise, ...
08:09:59 <morgan_orange> #info overall status summarized in wiki page
08:10:08 <morgan_orange> #link https://wiki.opnfv.org/functextnexttaks
08:10:19 <morgan_orange> no let's focus on tempest, vPing, ODL and Rally
08:10:24 <morgan_orange> #topic vPing
08:11:05 <morgan_orange> so still scenario failed odl_l3 (apex,fuel), onos (apex, joid)
08:11:25 <morgan_orange> but odl_l3 ok on fuel and onos ok on compass and fuel ....
08:11:48 <morgan_orange> jose_lausuch: may I action you to follow th vPing story on all the installers...
08:12:06 <jose_lausuch> https://wiki.opnfv.org/vping_brahmaputra_page
08:12:09 <jose_lausuch> sure
08:12:30 <morgan_orange> #action jose_lausuch vPing troubleshooting on remaining failed scenario
08:12:57 <morgan_orange> odl_l3 was OK on fuel but last run it was KO
08:13:30 <morgan_orange> that is why we have only 1 point on http://testresults.opnfv.org/proto/brahma/index-fuel.html when we select vPing duration
08:13:45 <morgan_orange> test done on 3/2
08:14:23 <morgan_orange> I am not sure that the problem is neither in the testcase nor in the installer but more on the POD config (as many errors in other test cases)
08:14:34 <morgan_orange> #topic Tempest
08:15:34 <morgan_orange> #info almost perfect on compass
08:15:47 <jose_lausuch> opnfv-jump-2 is failing due to some connectivity to public net
08:16:09 <morgan_orange> #info ~ 30 errors on apex and fuel, troubleshooting done by viktor_nokia
08:17:02 <morgan_orange> #info now logs should be properly pushed on the artifact
08:17:36 <morgan_orange> #action morgan_orange config of gsutils not correct on Orange POD 2 (but not supposed to be a CI POD) so results not pushed for the moment (login/password for gsutils)
08:18:01 <morgan_orange> viktor_nokia: Tempest run seems blocked on Orange POD2 at the moment..
08:18:38 <morgan_orange> as teh results are almost perfect on compass, I think we do not have to customize the list of tests...
08:18:41 <viktor_nokia> blocked? it seems to run...
08:20:17 <morgan_orange> ok let's wait a little bit 2016-02-09 07:53:23,734 - run_tempest - DEBUG - Executing command : rally verify start --tests-file /home/opnfv/repos/functest/testcases/VIM/OpenStack/CI/custom_tests/test_list.txt
08:20:52 <viktor_nokia> most burning problem in apex is so called "Worker missing" problem
08:21:17 <morgan_orange> yes
08:21:30 <morgan_orange> #info hot issue on apex or LF POD1?)
08:21:38 <morgan_orange> #info is the worker missing
08:22:00 <jose_lausuch> viktor_nokia: you filed a bug right?
08:22:14 <viktor_nokia> yes
08:22:16 <morgan_orange> not sure it is linked to Apex....but there is only 1 POD available. Even if as far as I understood POD7 is now dedicated to Apex
08:22:40 <morgan_orange> #action morgan_orange test workaround no parrallel on apex/LF POD1
08:22:53 <morgan_orange> I asked Dan yesterday but they were testing their patch for Be
08:23:01 <morgan_orange> so hopefully we can test it today
08:24:07 <morgan_orange> some of the tests, as far as I understood the logs, are due to lack of resources
08:24:56 <viktor_nokia> It is early to say, need to check result logs...
08:25:00 <morgan_orange> for joid, I suspect a config issue on POD as the same tests on Orange POD seems to be close to what we experience for apex and fuel
08:26:00 <morgan_orange> #action viktor_nokia troubleshooting Tempest, with at least access to logs on the artifact
08:26:13 <morgan_orange> but it seems that is logs are now available on http://artifacts.opnfv.org/
08:26:57 <morgan_orange> hmm to be checked, I can see Rally results from 8/2
08:27:16 <viktor_nokia> latest tempest.log is from 4/2
08:27:45 <morgan_orange> but still not tempest.log, there is a problem...
08:28:21 <morgan_orange> but it is probably normal...due to the regression we did not push results since the fix on gsutil
08:28:42 <morgan_orange> assuming that we are pushing rally, we should push temepst.log as soon as we produce it
08:28:50 <morgan_orange> #topic ODL
08:29:07 <morgan_orange> #info we have severla results 18/18, 15/18 but not always the same errors
08:29:45 <morgan_orange> initially I believed it was due to Li/Be but not so sure as Apex errors are not joid errors
08:30:01 <jose_lausuch> there should be 3 files for tempest results
08:30:04 <morgan_orange> #action morgan_orange troubleshoot ODL scenario to better undestand the difference of errors
08:30:06 <jose_lausuch> f_stdout = open(TEMPEST_RESULTS_DIR+"/tempest.log", 'w+')
08:30:06 <jose_lausuch> f_stderr = open(TEMPEST_RESULTS_DIR+"/tempest-error.log", 'w+')
08:30:06 <jose_lausuch> f_env = open(TEMPEST_RESULTS_DIR+"/environment.log", 'w+')
08:30:45 <morgan_orange> jose_lausuch: yes but we did not produce resutls in the last run, viktor_nokia provided a patch yesterday evening, I did not see it so the nighly run did not produce Tempest results
08:30:46 <morgan_orange> my bad
08:30:58 <morgan_orange> patch merge this morning and test in progress to validate on Orange POD 2
08:30:58 <viktor_nokia> BTW: Tempest result from Orange POD2: 2016-02-09 08:29:59,885 - run_tempest - INFO - Results: {'timestart': '2016-02-0907:53:24.971233', 'duration': 2192, 'tests': 210, 'failures': 18}
08:31:17 <morgan_orange> and it is validated
08:31:37 <morgan_orange> it is a big long compared to other run, but results are good
08:31:40 <jose_lausuch> ok
08:31:56 <morgan_orange> 88% of success...
08:32:16 <morgan_orange> but results not pushed to artifact because Orange POD has not the gsutils creds to push result (hence my action point)
08:32:29 <morgan_orange> #topic Rally
08:32:56 <morgan_orange> #info most of the scenario OK, heat scenario failed (but problem identified (static definition of ext network))
08:33:04 <jose_lausuch> apt-get.yum install gsutils :)
08:33:17 <morgan_orange> not only we need creds
08:33:33 <jose_lausuch> yep
08:33:38 <morgan_orange> but we will manage it if we are allowed to orange POD 2 is not a target CI labs
08:33:57 <morgan_orange> #info quota scenario OK on fuel but KO in apex
08:34:20 <morgan_orange> #info number of iteration has been set to 10 (instead of 20) to save time (-1h)
08:34:54 <morgan_orange> #action juhak patch for heat to use good ext network (all the installers have different names)
08:35:21 <juhak> ok
08:35:40 <jose_lausuch> I sent email yesterday proposing a change
08:35:59 <morgan_orange> shall we also do the summary at the end of the run as suggest by mail, I think we have the different elements except maybe the success rate that must be calculated. As there are different iteration, not sure which figure is the right one
08:36:39 <morgan_orange> so far, I did not create a page on the wiki, and rally is not in the dashboard (but pages are pushed into the artifact) , as there are lots of results..
08:36:45 <jose_lausuch> yes,  it would be great
08:36:56 <jose_lausuch> i can take it
08:37:21 <morgan_orange> http://artifacts.opnfv.org/logs/functest/huawei-us-deploy-bare-1/brahmaputra/2016-02-08_17-41-42/brahmaputra/rally/opnfv-nova.html
08:37:58 <morgan_orange> if juhak OK no problem to give you extra work... :)
08:38:22 <juhak> that's ok, no problem
08:38:44 <morgan_orange> #action jose_lausuch create a summary at the end of rally run
08:39:00 <morgan_orange> #action morgan_orange based on this summary pushed in DB add rally in dashboard
08:39:43 <morgan_orange> we should also find a way to reference the nice default page of Rally somewhere, at least in the documentation
08:40:03 <morgan_orange> and now question on live migration
08:40:14 <jose_lausuch> ok
08:40:32 <morgan_orange> shall we remove all the testcases related to live migration assuming we do not have always a shared storage to support it (as far as I understand it, it is not a genesis requirement)
08:41:21 <morgan_orange> #info shall we keep the testcase related to live migration assuming that our System Under Test does not explicitely support it
08:41:24 <jose_lausuch> ja...
08:41:37 <jose_lausuch> that was my question
08:41:40 <jose_lausuch> do we require live migration?
08:41:54 <jose_lausuch> if its not a genesis req, then I'd remove it
08:41:59 <jose_lausuch> if it is, then we keep it
08:41:59 <juhak> I think those cases (3) should be excluded if we can't expect system to support live migration
08:42:02 <morgan_orange> my suggestion we keep it (for next release) and we explain in the doc that this test will fail
08:42:21 <jose_lausuch> we can do that as well
08:42:34 <morgan_orange> so if people put in place a Pharos lab + shared storage => then he/she can test
08:42:56 <jose_lausuch> the ideal solution would be to detect if we have shared storage, if not, we dont trigger... but thats too complicated :D
08:43:05 <morgan_orange> or we comment it and in the doc we say if you want to test live migration, uncomment these 3 testcases
08:43:44 <morgan_orange> jose_lausuch: yes I mentioned that in the etherpad for C release, in the future we may think to more complex conditional scenario but let's discuss it in next sections
08:44:01 <jose_lausuch> ok
08:44:07 <jose_lausuch> I like last idea
08:44:22 <viktor_nokia> from tempest.conf: # Does the test environment support live migration available? (boolean # value) live_migration = false
08:44:34 <jose_lausuch> also, it will help to reach more % of success criteria if we remove it
08:45:05 <morgan_orange> ok so we remove it and update the doc to explain how to enable it for live migration (if the env support it)
08:45:08 <jose_lausuch> I like better viktor_nokia's way
08:46:05 <jose_lausuch> so, I guess it is true by default
08:46:51 <morgan_orange> #action juhak exclude 3 test cases related to live migration and document in the user guide how to uncomment if user would like to test it (or patch a dynamic mechanism based on tempest.conf parameter if time))
08:46:59 <viktor_nokia> Actually even tempest.conf generated by Rally (in Orange POD) contains: [compute-feature-enabled] live_migration = False
08:47:26 <morgan_orange> so it should be false everywhere...
08:47:37 <morgan_orange> I would suggest removal + doc and keep that for C release
08:48:08 <morgan_orange> ok only 12 minutes left
08:48:12 <morgan_orange> #topic Plugfest
08:48:33 <morgan_orange> #info I did not attend last dovetail meeting, but I plan to do it this week
08:48:59 <morgan_orange> #info Orange should organize the plugfest #2 in Europe, so I need to find resources on my side to follow this track
08:49:08 <morgan_orange> #undi
08:49:12 <morgan_orange> #undo
08:49:12 <collabot> Removing item from minutes: <MeetBot.ircmeeting.items.Info object at 0x1c529d0>
08:49:30 <jose_lausuch> do it in Laninon :)
08:49:31 <morgan_orange> for the moment Functest will "just" provide its container and the doc to use it
08:49:52 <morgan_orange> jose_lausuch: that is what I suggested...or I said I can deal with that if we organize it in Brittany... :)
08:50:34 <jose_lausuch> hehe
08:50:45 <morgan_orange> I do not know if referencing all the PODs/Hardware we run Functest on would be helpful...
08:51:09 <morgan_orange> I think it is more work for Pharos so we could link our run on PODX with Hardware Y, it could provide some itneresting information
08:51:21 <morgan_orange> as today POD are somehow abstracted but they are different
08:51:30 <morgan_orange> anyway though for post B release food
08:51:59 <morgan_orange> if you have any idea to support plugfest from a Functest perspective, do not hesitate...
08:52:03 <morgan_orange> #topic Berlin
08:52:13 <morgan_orange> #info CFP has been sent last week
08:52:39 <morgan_orange> a for Summit #1, I think we have materials to do some presentations
08:52:40 <jose_lausuch> I'd like to propose a presentation
08:53:30 <morgan_orange> Valentin and myself will probably propose something around vIMS (cloudify proposed something for Austin Openstack Summit with us as co-author, we will do the same for Berlin)
08:53:49 <jose_lausuch> ok
08:54:07 <jose_lausuch> I'll think about the topic
08:54:14 <jose_lausuch> but for sure something around functest :D
08:54:53 <morgan_orange> yes it can be on what we experience today (troubleshooting, testcases, links with upstream community ) or more prospective
08:55:30 <jose_lausuch> yes
08:55:32 <morgan_orange> we have also material for something on Rally for OPNFV and try to see how we could contribute upstream not only on the components (OS, ODL, ..) but alos on the tooling
08:55:55 <morgan_orange> viktor_nokia: juhak I think it could be interesting to suggets something with boris-42
08:56:08 <viktor_nokia> may be...
08:56:33 <morgan_orange> #topic Release C
08:56:57 <morgan_orange> #info etherpad updated
08:57:04 <morgan_orange> #link https://etherpad.opnfv.org/p/functest_release_C
08:57:32 <morgan_orange> we received sponatenous mail of people willing to join Functest. I asked them to join this weekly meeting
08:57:41 <morgan_orange> I do not know if there are some listening now?
08:58:13 <jose_lausuch> I filled that etherpad with an scheleton of the topics we can improve for release C
08:58:16 <SerenaFeng> I am listenibg
08:58:21 <jose_lausuch> if you see anything else, you can add it
08:58:48 <jose_lausuch> Just, put your name to see where the suggestions comes from :)
08:58:56 <morgan_orange> jose_lausuch: viktor_nokia juhak raghavendrachari do you know if you will still have resources to continue to work on Functest?
08:58:57 <SerenaFeng> But I am at home for chinese lunar year holiday
08:59:09 <morgan_orange> SerenaFeng: Happy new year BTW
08:59:09 <jose_lausuch> SerenaFeng: happy new year :)
08:59:24 <jose_lausuch> morgan_orange: what do you mean exactly?
08:59:50 <SerenaFeng> So I will attend the meeting officially after I return to shanghai
08:59:52 <morgan_orange> just to be sure that your management still allow you to continue to play in FuncTest
08:59:54 <SerenaFeng> Thank you
08:59:56 <jose_lausuch> I will do a doc update today, its draft on printed paper, but need to update it to the repo
09:00:07 <jose_lausuch> morgan_orange: sure :)
09:00:23 <juhak> morgan_orange: hoping to have more time for functest in the future
09:00:33 <viktor_nokia> me too
09:00:48 <jose_lausuch> :)
09:01:05 <jose_lausuch> we have a very good team
09:01:12 <morgan_orange> There are interesting challenges ahead, contribution upstream, multi sites testing, new test cases (I have an internship at the moment, he is looking at container based test cases..a vPing LXC for instance)
09:01:32 <morgan_orange> there are places for fun...
09:01:38 <raghavendrachari> yes
09:01:43 <jose_lausuch> there is room for improvements
09:01:55 <morgan_orange> and of course raghavendrachari we will rely on you for ODL improvement in C release...
09:02:02 <jose_lausuch> one of the ideas was to have functest executions at different levels
09:02:13 <raghavendrachari> ok
09:02:33 <morgan_orange> jose_lausuch: yes we can not keep on growing by adding test case one after the other...we will become too big...
09:02:51 <jose_lausuch> we need to define our own scenarios/set-of-tests
09:03:06 <morgan_orange> it would have been great to have a design summit in Europe (not sure most of use will attend the one pplanne din bay area in March)
09:03:10 <jose_lausuch> like yardstick
09:03:23 <viktor_nokia> BTW, does anybody from functest attend hackfest 14-15.3 ? (https://etherpad.opnfv.org/p/Q1'2016_Hackfest)
09:03:27 <jose_lausuch> not me , but I'll go to Berlin
09:03:31 <raghavendrachari> do we have any regress kind of testing ..
09:04:24 <morgan_orange> viktor_nokia: maybe we could suggest an informal Functest Hack fest somewhere in Europe
09:04:40 <viktor_nokia> in Helsinki :)
09:04:44 <morgan_orange> 1 day of workshop is not bad to define the roadmap
09:04:56 <juhak> +1
09:05:51 <jose_lausuch> I'd like to visit helsinki
09:05:54 <jose_lausuch> hehe
09:06:04 <morgan_orange> Ok I will ask my management, jose_lausuch would it be possible. I would suggest a informal Hackfest in Helsinki beginning of April?
09:06:15 <morgan_orange> or end of March
09:06:35 <jose_lausuch> we can also do it in spain, next to the beach :p
09:06:36 <morgan_orange> I have been there once for a Push To talk plugfest...and sailed in the bay..
09:06:36 <viktor_nokia> we need to ask our management :)
09:06:56 <morgan_orange> OK sorry raghavendrachari it is maybe a bit far for you :)
09:07:07 <morgan_orange> but juhak viktor_nokia jose_lausuch morgan_orange let's ask our management
09:07:14 <jose_lausuch> ok
09:07:19 <morgan_orange> #topic AoB
09:07:27 <morgan_orange> any other stuff you would like to share today?
09:07:51 <morgan_orange> ok we are already late
09:07:58 <morgan_orange> have a good week and keep on troubleshooting
09:08:02 <morgan_orange> thanks everyone
09:08:04 <morgan_orange> #endmeeting