08:00:08 <morgan_orange> #startmeeting Functest weekly meeting February 2nd
08:00:08 <collabot> Meeting started Tue Feb  2 08:00:08 2016 UTC.  The chair is morgan_orange. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:00:08 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:00:08 <collabot> The meeting name has been set to 'functest_weekly_meeting_february_2nd'
08:00:14 <morgan_orange> #info Morgan Richomme
08:00:18 <viktor_nokia> #info Viktor Tikkanen
08:00:19 <juhak> #info Juha Kosonen
08:00:54 <morgan_orange> #info agenda https://wiki.opnfv.org/functest_meeting
08:01:12 <morgan_orange> #info topic Action point follow up
08:01:21 <morgan_orange> #link http://ircbot.wl.linuxfoundation.org/meetings/opnfv-testperf/2016/opnfv-testperf.2016-01-26-08.00.html
08:01:49 <jose_lausuch> hi
08:01:50 <morgan_orange> #info hardware configuration check done on Huawei, Orange POD2 and LF POD2
08:01:55 <jose_lausuch> #info Jose Lausuch
08:01:59 <morgan_orange> hi jose_lausuch and raghavendrachari
08:02:11 <raghavendrachari> Hi
08:02:17 <raghavendrachari> have a nice day .!!
08:02:49 <jose_lausuch> :)
08:02:49 <morgan_orange> as far as I understand we excluded a problem tempest worker / hardware config of the jumphost
08:02:52 <jose_lausuch> what is the topic sorry?
08:03:08 <morgan_orange> just reviewing the action points of last week
08:03:30 <morgan_orange> we checked the HW config of the jumphost to see if it was the cause of the worker issue detected by viktor_nokia
08:03:49 <jose_lausuch> yes
08:03:53 <jose_lausuch> did we reach any conclussion?
08:04:42 <morgan_orange> I think we got the same config on LF POD1 and LF POD2 and had issue on LF POD1 not on LF POD2 so ...my understanding that this is not the cause of the error
08:04:49 <morgan_orange> but I may be wrong...
08:05:05 <jose_lausuch> and huawey pod had similar HW
08:05:17 <jose_lausuch> a lot of CPUs in all the cases...
08:05:21 <jose_lausuch> on the jumphost I mean
08:05:53 <jose_lausuch> Huawei sorry
08:05:55 <morgan_orange> #info juhak refactoring of volume management on rally-cert done
08:06:08 <morgan_orange> #info morgan_orange adaptation of oDL suite for JOID done
08:06:17 <jose_lausuch> juhak: rally shows always problems with live migration
08:06:29 <jose_lausuch> shall we say live migration of VMs is not supported ?
08:06:45 <morgan_orange> #info robot scenario path changed in ODL repo, but scenario did not change
08:06:57 <morgan_orange> #topic Release B
08:06:58 <juhak> live migration requires shared storage
08:07:14 <jose_lausuch> but we have ceph in theory...
08:07:28 <morgan_orange> #info refactoring of traces of Tempest and Rally in Jenkins done => nice results
08:07:33 <jose_lausuch> or we don't use it for ephemeral...
08:07:49 <morgan_orange> #info question on rally issue on live migration => shall these scenario be excluded?
08:07:59 <juhak> should we exclude live migration cases - for the time being?
08:08:42 <jose_lausuch> I don't know how the tests are performed, but I imagine it uses ephemeral (local disk) which I think it is not ceph, it's just local , but I'm not 100%
08:08:43 <jose_lausuch> sure
08:08:53 <jose_lausuch> but we might exclude them
08:08:54 <morgan_orange> if we know why it is failing...it makes sense...
08:09:09 <jose_lausuch> and just show the tests that are in tempest (if any) to prove that it fails
08:09:19 <jose_lausuch> and explain why
08:09:43 <jose_lausuch> May-meimei: can you check on Compass pods how is that implemented?
08:09:49 <morgan_orange> #info possible now to have a synthetic view of the different results
08:09:57 <morgan_orange> #link http://testresults.opnfv.org/proto/index-pod.html
08:09:57 <jose_lausuch> ceph for cinder? ceph for ephemeral?
08:10:07 <morgan_orange> #link http://testresults.opnfv.org/proto/index-scenario.html
08:10:18 <morgan_orange> #link http://testresults.opnfv.org/proto/index2.html
08:10:23 <morgan_orange> on Orange POD it is Ceph
08:10:38 <jose_lausuch> joid?
08:10:55 <morgan_orange> so tests should run but since yesterday we got regression, no Tempest tests are run (error saying that the config file cannot be found)
08:11:02 <morgan_orange> jose_lausuch: yes joid /orange POD2
08:11:23 <jose_lausuch> morgan_orange: ok, thanks, I see that live migration doesn't fail on orange pod2, but cinder stuff failed..
08:12:13 <morgan_orange> so if it is OK on some PODs, I suggest to keep the test and explains in the user guide...
08:12:17 <jose_lausuch> yes
08:12:20 <jose_lausuch> +1
08:12:26 <juhak> ok
08:12:36 <jose_lausuch> do we know why volume rally tests fail on that pod?
08:12:43 <jose_lausuch> test scenario CinderVolumes.create_and_attach_volume
08:12:48 <jose_lausuch> test scenario CinderVolumes.create_and_delete_volume
08:12:53 <jose_lausuch> test scenario CinderVolumes.list_volumes
08:12:57 <jose_lausuch> test scenario CinderVolumes.create_nested_snapshots_and_attach_volume3
08:13:01 <jose_lausuch> those are success 0%
08:13:04 <jose_lausuch> here https://build.opnfv.org/ci/view/functest/job/functest-joid-orange-pod2-daily-master/55/console
08:13:11 <jose_lausuch> to further investigate
08:13:14 <jose_lausuch> not now :)
08:13:36 <jose_lausuch> action someone with access to orange pod 2
08:13:38 <jose_lausuch> David? :)
08:13:58 <morgan_orange> I think David_Orange juhak viktor_nokia raghavendrachari and myself have access to this POD...
08:14:41 <raghavendrachari> yes am having access
08:14:43 <morgan_orange> #info regarding B release, still investigations mainly on vPing, vIMS
08:15:04 <juhak> yes, i can execute cinder cases manually in pod2 and check what's the problem
08:15:42 <jose_lausuch> juhak: ok, thanks. just things like that, cinder list, cinder create, snapshots, etc
08:15:52 <morgan_orange> #action juhak perform cinder test on Orange pod2
08:15:54 <jose_lausuch> attach
08:16:03 <morgan_orange> #info ODL suite on joid now looks like in apex (15OK /3 KO) I think that it is due to an update of the code, the tests did not follow => so I can change the test or just document it
08:16:09 <morgan_orange> #info ONOS tests OK
08:16:22 <morgan_orange> #info promise tests run
08:16:52 <morgan_orange> #info Tempest was relatively stable alert on regression on Orange POD2 (last 2 run => impossible to find the config) => I will have a look at it
08:17:05 <morgan_orange> #action morgan_orange Tempest on Orange POD2 where is the tempest.conf...
08:17:13 <jose_lausuch> morgan_orange: remember the tests that fail are the lats ones, delete stuff, that is what pbandzi did, not taken from the repo, maybe there is something to change
08:17:36 <morgan_orange> jose_lausuch: yes I will check that too
08:17:37 <jose_lausuch> yes, something fishy happens with tempest.conf
08:17:45 <jose_lausuch> I've seen it several times
08:17:47 <jose_lausuch> ok
08:18:01 <morgan_orange> #action morgan_orange ODL check the delete scenario from upstream repo (instead of tests in OPNFV repo)
08:18:22 <morgan_orange> what is strange is that it was very stable...I can do it manually to see what happens
08:19:02 <morgan_orange> #info Rally suite seems Ok for the release
08:19:25 <morgan_orange> I saw some errors on heat scenario but nothing critical for the release
08:19:42 <jose_lausuch> yep
08:19:47 <jose_lausuch> heat was failing in some cases too
08:19:49 <morgan_orange> for the vPing/SSH, I saw success results in Apex, Compass and Joid
08:19:58 <jose_lausuch> with ODL or ONOS?
08:20:05 <GeraldK_> are you IRC only? GTM tells me waiting for moderator.
08:20:19 <jose_lausuch> GeraldK_: yes, IRC only :)
08:20:38 <morgan_orange> for the vPing/SSH apex and joid with odl_l2
08:20:48 <morgan_orange> for compass also I think
08:20:49 <GeraldK_> okay. just wondered as the invitation had a GTM link
08:20:55 <juhak> <jose_lausuch>: I'll check those heat cases too on pod2
08:21:08 <jose_lausuch> morgan_orange: floating ips didnt work in Apex+ONOS
08:21:21 <morgan_orange> #action juhak checkheat cases on Orange POD2
08:21:40 <jose_lausuch> neither in Apex+ODL-l3
08:21:41 <morgan_orange> jose_lausuch: but it works once with apex now?
08:21:52 <morgan_orange> apex/odl_l2
08:22:11 <morgan_orange> so https://wiki.opnfv.org/vping_brahmaputra_page?&#vping is up to date?
08:22:11 <jose_lausuch> Apex+ODL-L2
08:22:12 <jose_lausuch> yes
08:22:25 <morgan_orange> comapss I saw something successfull with onos
08:22:38 <morgan_orange> very long 266s but it was ok
08:23:32 <jose_lausuch> morgan_orange: compass+ODL-l2 failed too   https://build.opnfv.org/ci/view/functest/job/functest-compass-huawei-us-deploy-bare-1-daily-brahmaputra/57/consoleFull
08:23:36 <jose_lausuch> Cannot establish connection to IP '192.168.10.101'. Aborting
08:23:45 <morgan_orange> yes the last success was compass/onos
08:24:09 <morgan_orange> Valentin_Orange: status on vIMS?
08:24:09 <jose_lausuch> yes https://build.opnfv.org/ci/view/functest/job/functest-compass-huawei-us-deploy-bare-1-daily-brahmaputra/56/consoleFull
08:24:37 <morgan_orange> #info vPing some scenario now working apex/odl_l2, compass/onos, joid/odl_l2....
08:24:40 <jose_lausuch> #action jose_lausuch push to DB if vPing test timeout
08:24:52 <morgan_orange> #info more run needed to update and confirm the status
08:26:10 <morgan_orange> #info as you know theoretically we have slot for troubleshooting
08:26:31 <Valentin_Orange> morgan_orange: ok on apex/odl_l2
08:26:49 <Valentin_Orange> morgan_orange: ok on fuel/odl_l2
08:27:08 <Valentin_Orange> morgan_orange: KO on  joid/odl_l2 ..
08:28:17 <morgan_orange> #link https://wiki.opnfv.org/vims_brahmaputra_page
08:28:29 <morgan_orange> #info progress on vIMS test case
08:28:49 <morgan_orange> #info vIMS OK on fuel and apex odl_l2, investigation in progress on joid and compass
08:29:08 <morgan_orange> #info reminder troubleshooting slots for Functest : LF POD1 (Apex) 18-20 UTC / LF POD2 (Fuel) 5-16 UTC, POD5 (Joid) 19-18 UTC, POD8 (Compass??) 13-15UTC, POD6 (joid), 19-20 UTC...
08:29:25 <morgan_orange> jose_lausuch: did you get access to POD5 and POD6?
08:29:40 <morgan_orange> just to know if it is now easier than LF POD access...
08:29:44 <jose_lausuch> morgan_orange: not yet, still waiting, the JIRA issue was changed to another person
08:29:49 <morgan_orange> :)
08:30:13 <morgan_orange> I think we can report that access to the POD is still an issue...and yesterday no POD was abvailable for testing...
08:30:26 <morgan_orange> we are supposed to be in testing phase with stable System Under Test....
08:30:35 <morgan_orange> no run of Fuel since 2 or 3 days..
08:30:51 <morgan_orange> The graphs are crual but they do not lie...
08:30:53 <jose_lausuch> still failed deployments sometimes :)
08:31:26 <morgan_orange> #info so to summarize on Functest for B release
08:32:10 <morgan_orange> #info vPing => need more scenario to check, recent progress in apex/odl_l2 and compass/onos but all combinations still not there
08:32:14 <leo_wang> vIMS is so not stable
08:32:22 <leo_wang> all kinds of problems
08:32:53 <morgan_orange> #info vPing_userdata need more scenario to confirm existing results (all old_l2 scenario OK)
08:33:32 <morgan_orange> #info ODL => check the 3 custom tests but OK on the different labs (just a test to check norhbound API of the Controller at the end)
08:33:50 <morgan_orange> #info ONOS => ok on all installers (need more run to confirm stability)
08:34:30 <jose_lausuch> good
08:34:45 <morgan_orange> #info vIMS OK on apex/fuel still issues with compass/joid,  complex test cases (more close to realistic needs of Telco)
08:34:56 <morgan_orange> leo_wang: how could we investigate more efficiently
08:35:06 <morgan_orange> would it be possible to grant access to Valentin_Orange?
08:35:15 <morgan_orange> we are lacking direct access to a compass lab
08:35:33 <morgan_orange> Intel POD8 has been assigned but can we use it? is there any Compass solution with a scneario running in this lab?
08:35:42 <leo_wang> the problem varies every time
08:36:11 <morgan_orange> a priori not with apex and fuel
08:36:16 <leo_wang> even failed to download cfy blueprint
08:36:23 <morgan_orange> network issue
08:37:01 <leo_wang> and create a venv may fail too
08:37:01 <morgan_orange> ok continue discuss it with Valentin_Orange offline but it would be great to have an access to the lab to help you
08:37:42 <leo_wang> :(
08:37:51 <leo_wang> ok
08:38:01 <morgan_orange> #info  promise OK
08:38:22 <morgan_orange> #info ovno a bit late but theoretically it is ready from a Functest/CI perspective
08:38:44 <morgan_orange> #info sdnvpn idem ovno wait for a run of the scenario to see if the tests are properly triggered
08:39:05 <morgan_orange> I think we should ask for a strategy on scenario on CI
08:39:22 <jose_lausuch> what do you mean?
08:39:29 <morgan_orange> CI slots have been defined..these slots must be used to run the different scenario
08:39:42 <morgan_orange> It is strange to have 1 run / day last week end for each installer...
08:39:49 <jose_lausuch> yes
08:39:54 <jose_lausuch> that is not acceptable
08:39:57 <morgan_orange> during the CI slot we should have at least 2 runs with 2 different scenario
08:40:26 <morgan_orange> #action morgan_orange propose scenario strategy for CI to get more deployment and then more tests
08:40:54 <morgan_orange> #info labs still not really ready for troubleshooting (no CI labs available yesterady)
08:41:37 <morgan_orange> #info doc has been updated, ChrisPriceAB split it to provide a global doc, everbody can still review it and patch it especially for the troubleshooting section
08:41:48 <morgan_orange> we have lots of things to say... :)
08:41:55 <morgan_orange> anythging else on B release?
08:42:24 <ChrisPriceAB> Once the docs are on master I should be able to put them in context.  Then we may need some editing.  But otherwise on track.
08:42:30 <morgan_orange> #action leo_wang provide access to Compass labs to Valentin_Orange for vIMS troubleshooting on compass
08:42:52 <jose_lausuch> ChrisPriceAB: aren't they on master already? what do you mean?
08:43:24 <ChrisPriceAB> They are, just saying I will have a central doc up soon.  Then whateer is on master will be pulled in.  (Patches will not)
08:43:42 <jose_lausuch> ChrisPriceAB: ok, thanks !
08:44:25 <morgan_orange> ok viktor_nokia juhak jose_lausuch Valentin_Orange ChrisPriceAB raghavendrachari leo_wang May-meimei anything else to add for B release...
08:44:55 <morgan_orange> #topic plugfest
08:44:56 <jose_lausuch> 1 small thing
08:45:00 <morgan_orange> #undo
08:45:00 <collabot> Removing item from minutes: <MeetBot.ircmeeting.items.Topic object at 0x1b71950>
08:45:16 <jose_lausuch> it seems everyone liked the new way of cleaning openstack defaults
08:45:25 <morgan_orange> oh yes...
08:45:30 <juhak> well done
08:45:33 <jose_lausuch> I'll merge https://gerrit.opnfv.org/gerrit/#/c/8927/1
08:45:41 <morgan_orange> ok
08:45:46 <viktor_nokia> :)  :)
08:46:06 <morgan_orange> if all the installers would have provide a consistent config, it would not have been needed but...we can see that it is useful...
08:46:20 <jose_lausuch> maybe for the next release we can have 2 options
08:46:38 <jose_lausuch> cleanup everything, or just cleanup the things that were there before the tests
08:46:41 <morgan_orange> let's discuss in the last 10 minutes discussions on next release...
08:46:45 <morgan_orange> #topic plugfest
08:46:50 <jose_lausuch> sure
08:47:03 <morgan_orange> #info as you know a plug fest is on track. it will be organized in Colorado in May (Cable Labs)
08:47:09 <morgan_orange> question 1: who will attend?
08:47:24 <morgan_orange> I mean from Functest contributor;..
08:47:37 * jose_lausuch doesn't...
08:47:50 <morgan_orange> I will try to follow the discussion on dovetail on this topic but not easy to find time
08:48:04 <morgan_orange> I am not sure we are mandatory for such plugfest but our tests will be...
08:48:09 <juhak> Nokia is planning to participate PlugFest. don't know yet if Viktor or me will be there
08:48:52 <morgan_orange> I think Orange will probably also participate but not clear who, all the more as we are supposed to organize a plugfest in Q3 in France
08:49:27 <morgan_orange> so is userguide + container + adapation of the graphs enough as Functest contribution to this plugfest?
08:49:37 <morgan_orange> or do you see stuff we should have to do from no to May
08:49:45 <morgan_orange> specifically for such event?
08:50:22 <morgan_orange> #info Nokia and Orange will probably participate (need confirmation, people not designated at the moment)
08:50:47 <morgan_orange> #info question what can we bring as input for such event
08:50:50 <jose_lausuch> I think that event is more focus on NFV point of view right?
08:50:58 <jose_lausuch> something maybe more important for yardstick guys
08:51:40 <morgan_orange> yep even if running our basic suites gives already a good feedback...and provide some elements of comparison...
08:52:07 <morgan_orange> but OK the idea is not to answer today, I will keep a small point also next weeks to discuss that and I will try to follow a little bit more dovetail
08:52:11 <ChrisPriceAB> I think there will be some YardStick people there and Fuel.
08:52:19 <morgan_orange> #topic Release C
08:52:36 <ChrisPriceAB> But functest has a bunch of troubleshooting skils that would be applicable so don't not go... :D
08:52:59 <morgan_orange> #link https://etherpad.opnfv.org/p/functest_release_C
08:53:23 <jose_lausuch> ChrisPriceAB: :) to what level troubleshooting fits in that event?
08:53:37 <morgan_orange> #info for the moment it is a white page, I see already some possible actions/improvements and I have questions
08:54:15 <ChrisPriceAB> plugging in new components, running our software on new hardware.  There is likely trouble in those waters...  (back on topic jose_lausuch)
08:54:17 <jose_lausuch> morgan_orange: I will fill some info, some ideas
08:54:39 <jose_lausuch> ChrisPriceAB: ok, I see, thanks
08:54:40 <morgan_orange> for ODL, I do not know if we should keep it inside or externalize it (as ONOS)
08:55:06 <morgan_orange> I think it is time to try to give back to rally and to contribute directly on rally  to define Telco KPI
08:55:15 <morgan_orange> question who goes to Austin for the OpenStack summit...
08:55:25 <morgan_orange> I think it would be easier to do that in Barcelona...
08:55:28 <jose_lausuch> morgan_orange: I will add some titles (blocks) for each component, so people can add ideas in an ordered way
08:55:48 <jose_lausuch> Barcelona is closer ;)
08:55:48 <morgan_orange> yes I think we should consider
08:56:01 <morgan_orange> not in time..
08:56:10 <morgan_orange> - existing testcases (evolution, ...)
08:56:20 <morgan_orange> - nex test cases (sync with Intel and new VNF)
08:56:33 <morgan_orange> - use of orchestrator?
08:56:34 <jose_lausuch> yep
08:56:42 <jose_lausuch> something to be discussed of course
08:56:44 <morgan_orange> - better analytics of our results (we are building a mine)
08:57:04 <morgan_orange> - move to current home API to something like ELK ?
08:57:12 <jose_lausuch> ELK?
08:57:22 <morgan_orange> Elastic Search / Logstash / Kibana
08:57:22 <jose_lausuch> ah ok
08:57:28 <jose_lausuch> yep
08:57:46 <morgan_orange> discuss when we should onboard or externalize companion test cases
08:57:47 <jose_lausuch> shall we everyone take the action to write ideas?
08:58:09 <morgan_orange> #info etherpad open for Functest Release C
08:58:18 <morgan_orange> #info  we should consider:
08:58:36 <morgan_orange> #info  - existing testcases (evolution, in&out)
08:58:45 <morgan_orange> #info next test cases (sync with Intel and new VNF)
08:58:54 <morgan_orange> #info role of orchestrator for testing
08:59:08 <morgan_orange> #info -upstream contribution( E.g. Rally)
08:59:17 <morgan_orange> #info - analytics?
08:59:35 <morgan_orange> #info moving from home made API + Mongo to ELK?
08:59:52 <morgan_orange> #action evrybody complete this etherpad
09:00:02 <morgan_orange> #link https://etherpad.opnfv.org/p/functest_release_C
09:00:27 <morgan_orange> we will have more time after C release, but we will probably not see in real before Berlin, which will be too late for new feature definition
09:00:52 <morgan_orange> we need to have a clear view of our vizion for March/April => idea is to define the figure as we did for brahmaputra
09:01:15 <morgan_orange> https://wiki.opnfv.org/functextnexttaks
09:01:27 <morgan_orange> The figure has not been changed since last July
09:01:43 <morgan_orange> and it is was not fully stupid...
09:01:53 <morgan_orange> #topic AoB
09:01:57 <morgan_orange> we are 1 minute late
09:02:12 <morgan_orange> anything you want to share before ending the meeting?
09:02:22 <jose_lausuch> nop
09:02:27 <morgan_orange> I want to share one => vote for jose to quartely awards...
09:02:29 <jose_lausuch> everything ok
09:02:54 <jose_lausuch> hahaha its not needed, you are too nice :)
09:02:59 <morgan_orange> we can vote also to viktor_nokia: juhak Valentin_Orange for their contribution
09:03:13 <morgan_orange> and also May-meimei of course
09:03:41 <jose_lausuch> I'm already the Rock'n Tester...  ChrisPriceAB^
09:04:14 <morgan_orange> ok so have a good week, lots of work ahead...meaning that such system needs still some testing...
09:04:27 <morgan_orange> #endmeeting