08:00:42 <jose_lausuch> #startmeeting Functest weekly meeting January 31th 2017
08:00:42 <collabot> Meeting started Tue Jan 31 08:00:42 2017 UTC.  The chair is jose_lausuch. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:00:42 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:00:42 <collabot> The meeting name has been set to 'functest_weekly_meeting_january_31th_2017'
08:00:56 <jose_lausuch> #topic role call
08:00:56 <jose_lausuch> #chair morgan_orange
08:00:56 <jose_lausuch> #info Jose Lausuch
08:00:56 <collabot> Current chairs: jose_lausuch morgan_orange
08:01:13 <morgan_orange> #info morgan Richomme
08:01:16 <jose_lausuch> who is here today?
08:01:18 <juhak> #info Juha Kosonen
08:01:19 <rohitsakala> #info rohitsakala
08:01:26 <ashishk> #info ashishk
08:01:29 <jose_lausuch> good
08:01:37 <hideyasu> #info hideyasu
08:01:47 <jose_lausuch> I think our chinese colleagues are on holidays
08:01:56 <morgan_orange> hideyasu: 今日は
08:02:17 <hideyasu> morgan , oh hello !
08:02:21 <jose_lausuch> morgan_orange: I didnt know about your language skills :)
08:02:29 <jose_lausuch> #topic Action point follow-up
08:02:35 <morgan_orange> my google friends is not bad...
08:02:39 <jose_lausuch> #info AP: ashishk sinc with Mark Beier. Email sent, waiting for Mark's answers and start working together.
08:02:45 <jose_lausuch> ashishk: anything else?
08:03:03 <ashishk> jose_lausuch: no, I am waiting for his response to update the slides
08:03:09 <jose_lausuch> ok, good
08:03:23 <jose_lausuch> I have some info regarding apex and v3
08:03:24 <jose_lausuch> #info AP: jose_lausuch talk to trozet about openstack version vs keystone version. Can we use v3 as all the others? --> Tim said that v3 is supported, but not by default. Tempest test cases with v3 pass but Apex depends on TripleO releases and they will move to v3 in the next release, so for Danube we will have still v2 and E-river v3 most probably.
08:04:07 <jose_lausuch> #info AP jose_lausuch. create more than 1 fuel deployment. Resources are ok (enough disk and ram). DONE
08:04:13 <morgan_orange> so what is teh status on keystone v3
08:04:19 <jose_lausuch> for apex?
08:04:21 <morgan_orange> apex => v2, other installers v3 by default?
08:04:30 <jose_lausuch> ya
08:04:46 <morgan_orange> ok so fro promise (fuel or joid) they must do an adaptation
08:04:46 <jose_lausuch> well, all of them support v2 of course
08:04:52 <Cristina_> #info Cristina Pauna (ENEA)
08:04:57 <morgan_orange> ok so it should still work anyway...
08:05:05 <jose_lausuch> ya
08:05:08 <morgan_orange> ok
08:05:12 <jose_lausuch> I think v2 is gonna be deprecated soon
08:05:15 <jose_lausuch> in Ocata maybe
08:05:18 <morgan_orange> I can add the statement in the user guide somewhere
08:05:21 <jose_lausuch> and all will use v3 by default
08:05:30 <jose_lausuch> yes, it would be useful, thanks
08:05:43 <morgan_orange> #action morgan_orange indicate keystone v3 status in documentation for Danube
08:05:54 <jose_lausuch> :)
08:06:02 <jose_lausuch> #info AP jose_lausuch. create 3 scenarios in our server: nosdn, odl_l2, odl_l3 DONE
08:06:06 <bertys_> jose_lausuch, morgan_orange: keystone v2 is already deprecated...
08:06:17 <bertys_> #link http://developer.openstack.org/api-ref/identity/v2/ ...
08:06:22 <jose_lausuch> bertys_: tell our apex friends :)
08:06:45 <bertys_> jose_lausuch: they probably know already
08:07:05 <jose_lausuch> is there an impediment for us, by using v2?
08:07:13 <jose_lausuch> I mean, I don't want to force anyone to do soemthing
08:07:26 <jose_lausuch> but I agree all the installers should use the same
08:07:43 <jose_lausuch> we can't impose an installer to use version XYZ
08:08:09 <jose_lausuch> #info AP jose_lausuch add topic on feature tests refactor. DONE
08:08:16 <jose_lausuch> #info AP LindaWang check path issue with copper on apex virtual pod
08:08:24 <jose_lausuch> do you know he she has done it?
08:08:30 <jose_lausuch> I saw some commits..
08:08:46 <morgan_orange> yes but not sure it has been tested yet
08:08:52 <morgan_orange> copper still fails on apex
08:08:58 <morgan_orange> not sure it is a path issue
08:09:01 <jose_lausuch> #action re-check: LindaWang check path issue with copper on apex virtual pod
08:09:05 <morgan_orange> I can quickly check on the virtual pod
08:09:12 <jose_lausuch> ok
08:09:20 <jose_lausuch> I provided a bare metal to btyan
08:09:24 <jose_lausuch> I hope he can also work on that
08:09:33 <morgan_orange> yes probably the best option
08:09:46 <morgan_orange> but I can check if the issue is on the integration, once copper is launched it is up to him
08:09:55 <jose_lausuch> ya, that is right
08:10:01 <jose_lausuch> integration is on us
08:10:01 <jose_lausuch> well
08:10:08 <jose_lausuch> that's not 100% true, but ok :)
08:10:16 <jose_lausuch> that's why we have a dev guide
08:10:34 <jose_lausuch> anyway, what about your ap with exit conditions in snaps
08:11:14 <morgan_orange> I created a JIRA
08:11:25 <morgan_orange> and I sent a mail before to steve
08:11:42 <jose_lausuch> #info AP morgan_orange contact steve to discuss exit conditions in SNAPS. -> Jira created and contacted Steve
08:11:47 <jose_lausuch> ok, thanks
08:11:47 <morgan_orange> #link https://jira.opnfv.org/browse/FUNCTEST-713
08:11:56 <morgan_orange> it is maybe linked
08:12:05 <morgan_orange> in case of SNAPS error, the FAIL status is not reported
08:12:09 <jose_lausuch> good, any other AP?
08:12:15 <jose_lausuch> aha
08:12:19 <morgan_orange> that is why the page http://testresults.opnfv.org/reporting/functest/release/master/index-status-compass.html is misleading
08:12:25 <morgan_orange> SNAPS should be failed
08:12:30 <jose_lausuch> you mean, code failure or openstack failure?
08:12:32 <morgan_orange> but when fail status never reported to the DB
08:12:36 <morgan_orange> I created also a JIRA for that
08:12:41 <jose_lausuch> ok
08:12:53 <morgan_orange> https://jira.opnfv.org/browse/FUNCTEST-714
08:12:56 <jose_lausuch> #info JIRA created for reporting a failure to DB in SNAPs
08:13:01 <morgan_orange> #link https://jira.opnfv.org/browse/FUNCTEST-714
08:13:12 <jose_lausuch> ok
08:13:19 <jose_lausuch> I hope Steven can work on that soon
08:13:32 <jose_lausuch> and I hope we can have a repo in opnfv soon as well
08:13:40 <jose_lausuch> ok
08:13:42 <jose_lausuch> next topic
08:13:48 <jose_lausuch> #topic Troubleshooting status (short update)
08:13:56 <morgan_orange> regarding snaps I think we could move connection_check and api_check in healthcheck category
08:14:07 <morgan_orange> or shall we wait for joid...
08:14:15 <jose_lausuch> I will summarize the status for each installer , using the email from morgan yesterday (thanks for it!)
08:14:16 <morgan_orange> it seems OK in apex, compass and fuel
08:14:32 <jose_lausuch> morgan_orange: we can try this week already, maybe we wait 2 days
08:15:03 <jose_lausuch> it will impact also apex verify jobs
08:15:14 <jose_lausuch> so, we need to be careful
08:15:20 <jose_lausuch> but we can do it this week I think
08:15:21 <morgan_orange> #action morgan_orange move api_check/connection_check in healtcheck on Thursday + remove historical healtcheck
08:15:50 <jose_lausuch> #info there is something useful in healtcheck which is checking if the VMs get an IP from the DHCP agent
08:16:00 <jose_lausuch> #action check with Steven if that is implemented in SNAPS
08:16:10 <jose_lausuch> it is a good gate
08:16:17 <morgan_orange> ok I think it makes also sense to keep our vpings which are quiete robust...
08:16:20 <jose_lausuch> if the VMs dont get an IP, nothing else will work
08:16:24 <jose_lausuch> yes
08:16:25 <morgan_orange> and maybe add vping_snaps
08:16:30 <morgan_orange> in //
08:16:32 <jose_lausuch> yes
08:16:41 <morgan_orange> we will decide in E if we shall remove the historical ones?
08:16:44 <jose_lausuch> I would like to have first a vping snaps before converting them
08:16:52 <jose_lausuch> good idea
08:17:13 <jose_lausuch> well, we can still do it for vping maybe in Danube....
08:17:15 <jose_lausuch> let's seee
08:17:28 <jose_lausuch> if we see that snaps tests are working soon, we can take the risk
08:17:35 <jose_lausuch> but let's discuss it further another day
08:17:40 <jose_lausuch> during next 2 weeks maybe
08:17:50 <jose_lausuch> #info APEX
08:17:55 <jose_lausuch> #info Tests that are ok: Healthcheck, vping_ssh/userdata, rally, security_scan
08:18:01 <jose_lausuch> #info Test that fail in one or more scenarios: doctor, copper, tempest_smoke (by just 1 error), snaps_smoke
08:18:18 <jose_lausuch> #info Compass
08:18:26 <jose_lausuch> #info healthcheck fails on almost all the scenarios. It only works on odl_l2
08:18:37 <jose_lausuch> #info rally fails and 5 tempest errors  on odl_l3-nofeature
08:18:45 <jose_lausuch> #info Daisy
08:18:50 <jose_lausuch> #info not integrated with Functest yet
08:19:01 <jose_lausuch> and I sent an email on friday...
08:19:04 <jose_lausuch> no answer...
08:19:16 <jose_lausuch> let's see what happens here
08:19:23 <jose_lausuch> #info Fuel
08:19:31 <jose_lausuch> #info healtcheck passes only in nosdn-ovs scenario, but not in the base nosdn-nofeature scenario
08:19:45 <jose_lausuch> you see? this has been detected thanks to the healthcheck, dhcp check
08:19:55 <jose_lausuch> #info already contacted with the Fuel team and are in the progress of troubleshooting this on the ericsson-pods
08:20:18 <jose_lausuch> #info Joid
08:20:22 <jose_lausuch> #info healtcheck fails
08:20:41 <jose_lausuch> #info some concerns from Functest team:
08:20:46 <jose_lausuch> #info very few installers have good results...
08:20:49 <morgan_orange> could be linked to the patch to support SNAPS, I hesitated to revert it...
08:21:09 <jose_lausuch> what patch/
08:21:10 <jose_lausuch> ?
08:21:47 <morgan_orange> the patch with strange character at teh beginnign of the rc file, since this patch everything fails (but it is maybe a coincidence..;I sent a mail to narinder)
08:22:10 <morgan_orange> there is another one on Maas 2.0
08:22:25 <jose_lausuch> ah ya
08:22:26 <jose_lausuch> right
08:22:30 <morgan_orange> I will try to follow up on joid
08:22:31 <jose_lausuch> but it was corrrected
08:22:44 <morgan_orange> https://gerrit.opnfv.org/gerrit/#/c/27595/
08:23:11 <jose_lausuch> ok
08:23:21 <jose_lausuch> #action morgan_orange follow up on JOID progress
08:23:25 <jose_lausuch> #info ocl scenario run but no dedicated test suite
08:23:25 <morgan_orange> another point for juhak on Tempest, for all the scenarios, there are 40-50 skipped tests systematically
08:23:31 <jose_lausuch> yes
08:23:32 <jose_lausuch> right
08:23:37 <jose_lausuch> not sure why
08:23:48 <juhak> on all scenarios?
08:23:52 <jose_lausuch> mostly
08:24:11 <jose_lausuch> see email from morgan yesterday, there are some links
08:24:21 <jose_lausuch> can I action you on that?
08:24:21 <juhak> ok
08:24:26 <juhak> yes
08:24:36 <jose_lausuch> #action juhak investigate why tempest skips 40-50 tests
08:24:43 <jose_lausuch> maybe because of using the new tempest...
08:24:46 <jose_lausuch> new versions
08:24:48 <jose_lausuch> I Dont know
08:24:53 <jose_lausuch> #info no news from Daisy after MS5 => no Functest in Daisy for Danube?
08:24:57 <morgan_orange> there are also some errors on Rally on some scenarios only (nova, heat, requests)
08:25:07 <morgan_orange> not systemati
08:25:14 <jose_lausuch> there was a patch about rally
08:25:17 <jose_lausuch> not sure if it will fix that
08:25:34 <jose_lausuch> for example, for storage
08:25:39 <morgan_orange> there was also a pathc on Tempest (version update)
08:25:43 <jose_lausuch> something needed in tempest.conf
08:25:58 <jose_lausuch> #info a lot of features/scenarios not run on jenkins yet
08:26:17 <jose_lausuch> anything to add to this topic?
08:26:29 <morgan_orange> no apex run since the 28th
08:26:35 <morgan_orange> scoring will decrease soon.. :)
08:26:55 <morgan_orange> regarding the release meeting, i think it is reasonable to raise an orange/red flag
08:27:00 <morgan_orange> we reached MS5
08:27:11 <morgan_orange> and today there is only fuel HA runs on Danube
08:27:19 <morgan_orange> or almost
08:27:37 <morgan_orange> installer integration should have been more reliable since 1 month, no?
08:27:49 <jose_lausuch> I will
08:27:58 <jose_lausuch> fuel HA?
08:27:58 <morgan_orange> so withouth surprise, feature integration did not really start
08:28:04 <jose_lausuch> not apex HA ?
08:28:14 <morgan_orange> not since the 28th of January
08:28:18 <jose_lausuch> ah ya, right
08:28:22 <jose_lausuch> but there was something before
08:28:27 <jose_lausuch> I'll contact trozet
08:28:37 <jose_lausuch> but we can raise a flag to David
08:28:43 <morgan_orange> yes as there was something on joid...but at the moment it is not very stable :)
08:28:57 <morgan_orange> on compass non HA seems fine but not HA not
08:28:58 <jose_lausuch> it has been hard for everyone, no one has been on time
08:29:02 <morgan_orange> shall I remove non HA from reporting
08:29:10 <morgan_orange> it is maybe a bit misleading
08:29:14 <jose_lausuch> yes
08:29:27 <morgan_orange> #action morgan_orange remove non HA to draw daily reporting
08:29:54 <jose_lausuch> we have a topic for feature integration :)
08:30:00 <jose_lausuch> and I will use again your long email from yesterday
08:30:01 <jose_lausuch> :p
08:30:04 <jose_lausuch> but first
08:30:07 <jose_lausuch> #topic Danube documentation
08:30:20 <jose_lausuch> #info 2 patches from morgan to update the user and dev guide
08:30:26 <jose_lausuch> #link https://gerrit.opnfv.org/gerrit/#/c/27385/
08:30:33 <jose_lausuch> #link https://gerrit.opnfv.org/gerrit/#/c/27665/
08:30:38 <jose_lausuch> I still need to review the dev guide
08:30:42 <morgan_orange> and one for config guide
08:30:44 <morgan_orange> #link https://gerrit.opnfv.org/gerrit/#/c/27247/
08:30:44 <jose_lausuch> but provided some comments to the user guide
08:30:49 <jose_lausuch> ah, I missed that sorry
08:30:54 <jose_lausuch> so
08:31:09 <jose_lausuch> our Nokia colleagues left very good consistent guides
08:31:11 <morgan_orange> I did not take into account the possible move of snaps tests into healthcheck
08:31:17 <jose_lausuch> just a few modifications are needed :)
08:31:24 <jose_lausuch> shall we?
08:31:31 <jose_lausuch> I think this is what we want for Danube
08:31:32 <jose_lausuch> or?
08:31:37 <morgan_orange> not yes, not until it is done:)
08:31:44 <jose_lausuch> haha ok
08:32:04 <jose_lausuch> fair enough
08:32:04 <morgan_orange> and I agree I just made minor adaptation the baseline provided by Colum and Juha is very good
08:32:08 <jose_lausuch> we can update it later
08:32:52 <jose_lausuch> anything you want to raise about documentation?
08:32:57 <morgan_orange> there is also a document for the testing community in progress where we reference Functest documentation
08:32:57 <jose_lausuch> you have done all the job this time :)
08:33:08 <morgan_orange> release note is not written yet... :)
08:33:14 <jose_lausuch> that's up to me
08:33:17 <jose_lausuch> but not yet
08:33:21 <morgan_orange> and I will suggest to the testing group to redo what we did in Colorado
08:33:25 <morgan_orange> a cross project review
08:33:44 <morgan_orange> it was very useful
08:33:49 <jose_lausuch> yes
08:34:07 <jose_lausuch> #action morgan_orange suggest a cross project review of the documentation to the testing group
08:34:18 <jose_lausuch> sofia has also a server for documentation testing
08:34:34 <jose_lausuch> so she might also try to execute the steps in our documentation :)
08:35:07 <jose_lausuch> which will be another good  source of feedback
08:35:12 <morgan_orange> I will also maybe re raised what you mentioned in Colroado...could we have review by people skilled in such documentation
08:35:38 <jose_lausuch> ya, you can try this time :D
08:36:06 <morgan_orange> maybe one modification on our doc would be to be more systematic on the config, we use apex/compass/fuel/joid tips
08:36:15 <jose_lausuch> I think, most companies have some contractors that are experts in "language" and documenting
08:36:23 <jose_lausuch> they basically change all what you have written...
08:36:57 <morgan_orange> I will take your comment into account
08:37:10 <morgan_orange> I suggest then to merge, we will repatch later
08:37:14 <jose_lausuch> I could imagine LF could provide something similar
08:37:20 <jose_lausuch> morgan_orange: ok
08:37:21 <morgan_orange> yes I imagien so
08:37:27 <jose_lausuch> let me review first the config and dev guides
08:37:31 <jose_lausuch> havent had the time yet
08:37:33 <morgan_orange> the difficulty is the consistency accross the project
08:37:50 <morgan_orange> the level of description is very different from one project to another...
08:38:03 <jose_lausuch> morgan_orange: don't ask too much... that will come but dont think we can have it now
08:38:06 <morgan_orange> anyone is invited to review....
08:38:18 <morgan_orange> Serena and helan already did but any new comments are wecome
08:38:22 <jose_lausuch> we could raise that to the docs team
08:38:46 <jose_lausuch> well, if they +2ed, it will be very good already
08:38:58 <jose_lausuch> #topic Feature project integration status
08:39:10 <jose_lausuch> so, using your email (again) :)   :
08:39:14 <jose_lausuch> #info doctor run (apex) but FAIL
08:39:19 <jose_lausuch> #info copper run (apex) but FAIL
08:39:23 <jose_lausuch> #info promise run (fuel) but FAIL --> Morgan working on it with Promise guys
08:39:27 <jose_lausuch> #info parser run (fuel)  but FAIL
08:39:32 <jose_lausuch> #info ovs run (fuel) but FAIL
08:39:36 <jose_lausuch> #info NOT run: moon, bgpvpn, odl-sfc, onos-sfc, ons, domino, kvm
08:39:42 <jose_lausuch> #info no recent news from: opera, orchestra, movie, models, ...
08:39:51 <jose_lausuch> #info NetReady PTL contacted me to start creating a test case.
08:39:58 <jose_lausuch> but no scenario in CI
08:40:34 <jose_lausuch> #info bgpvpn scenario activation is waiting for merge
08:40:52 <jose_lausuch> #info sfc scenario already discussed with fuel and apex to activate it in CI asap
08:40:58 <jose_lausuch> for the others I don't have news
08:41:40 <morgan_orange> Does the Release manager have?
08:41:41 <jose_lausuch> so, M5 is not ready for most of the features
08:41:58 <jose_lausuch> not many people answered to M5 questions from David
08:42:12 <jose_lausuch> M5 doesn't apply to us
08:42:24 <jose_lausuch> but I don't know what's gonna happen
08:42:27 <jose_lausuch> :)
08:42:39 <jose_lausuch> 1.5 months to realease
08:43:36 <morgan_orange> look like more Brahmaputra than Colorado...
08:44:28 <jose_lausuch> yes
08:44:42 <morgan_orange> shall we create Jira and try to associate a feature project integration with someone from Functest?
08:44:52 <morgan_orange> first I think we need the commitments from the release manager
08:44:58 <morgan_orange> then we could do that...
08:45:10 <jose_lausuch> you mean that each of us take responsability for feature integrations?
08:45:22 <morgan_orange> there are also strange things on the scenarios, I can see joid/k8 scenarios trying to launch Functets ...no sense
08:45:30 <jose_lausuch> ya
08:45:35 <morgan_orange> and no news from fido scenarios
08:45:59 <morgan_orange> we will not take the responsability of everything
08:46:16 <morgan_orange> but if the feature is confirmed, it is a way to indicate to the feature project a Functest counterpart
08:46:18 <morgan_orange> to help
08:46:32 <morgan_orange> in theory it is not needed
08:46:38 <morgan_orange> as they are not so many new features...
08:46:55 <morgan_orange> but as we do not know what will be in and/or out
08:46:56 <jose_lausuch> ok
08:47:04 <jose_lausuch> right
08:47:06 <morgan_orange> and maybe it is ready but no CI scenarios available
08:47:09 <morgan_orange> i do not know
08:47:51 <morgan_orange> the milestone sshould have been shifted as the previous one...and the release date as well...
08:48:17 <morgan_orange> BTW hideyasu do you still plan to integrate the vrouter suite for Danube?
08:48:45 <hideyasu> yes i do
08:49:11 <morgan_orange> OK, you can use the VNF abstraction if needed, and do not hesitate to contact me if I can help
08:49:36 <hideyasu> thank you so much.
08:50:16 <jose_lausuch> I hope we can have it for danube :)
08:51:04 <morgan_orange> another question regardung Helen patch on Conf management. In theory as we are post M5, we should focus on CI/troubleshooting and keep such patch for E release, no?
08:52:01 <jose_lausuch> https://gerrit.opnfv.org/gerrit/#/c/27603/3 ?
08:52:02 <jose_lausuch> this?
08:52:11 <jose_lausuch> let's move to aob
08:52:13 <jose_lausuch> #topic AoB
08:53:14 <morgan_orange> jose_lausuch: yes and in general...can we still refactor or say that any refactoring is now for E release?
08:53:50 <jose_lausuch> morgan_orange: we should only pass the testcasebase adaptation for those that are missing
08:54:03 <jose_lausuch> but major refactorings should not happen now, as they can break CI
08:54:10 <jose_lausuch> and we should have an stable framework
08:54:18 <jose_lausuch> and start thinking about fixing bugs
08:54:21 <morgan_orange> ok we may info that
08:54:24 <jose_lausuch> instaed of adding new functionality
08:54:30 <jose_lausuch> you can info it :)
08:54:39 <morgan_orange> #info we should only pass the testcasebase adaptation for those that are missing
08:54:46 <morgan_orange> #info but major refactorings should not happen now, as they can break CI
08:54:59 <morgan_orange> #info we should have an stable framework and start thinking about fixing bugs
08:55:28 <morgan_orange> #info no new feature in Functest framework for Danube
08:56:42 <morgan_orange> #info green light from LF to ask bitergia to manage Dashboard (consuming test API)
08:57:21 <morgan_orange> #info Plugfest officially announced - will be in paris from the 24 to 27 => good occation for European contributors => organize a meetup here for E?
08:57:31 <jose_lausuch> yes!
08:58:09 <jose_lausuch> let's do that
08:58:46 <morgan_orange> good luck for the release meeting :)
08:59:03 <jose_lausuch> ya, I always get a lot of APs :)
08:59:06 <jose_lausuch> thanks
08:59:15 <jose_lausuch> looking forward for the plugfest
08:59:23 <jose_lausuch> finally I'll see your land :p
08:59:27 <morgan_orange> yes but it is a bit strange, sometimes there is a confusion between Functest and release management
08:59:33 <morgan_orange> Functets is not the release management project...
08:59:53 <jose_lausuch> it's not
09:00:00 <jose_lausuch> but it seems we generate trust :)
09:00:24 <jose_lausuch> ok, need to jump to another meeting
09:00:27 <jose_lausuch> thanks all
09:00:29 <jose_lausuch> have a good day
09:00:31 <jose_lausuch> #endmeeting