08:00:20 <morgan_orange> #startmeeting Functest weekly meeting January 5th
08:00:20 <collabot`> Meeting started Tue Jan  5 08:00:20 2016 UTC.  The chair is morgan_orange. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:00:20 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:00:20 <collabot`> The meeting name has been set to 'functest_weekly_meeting_january_5th'
08:00:34 <morgan_orange> #info Morgan Richomme
08:00:38 <viktor_nokia> #info Viktor Tikkanen
08:00:39 <morgan_orange> Happy new year everybody
08:00:40 <juhak> #info Juha Kosonen
08:01:03 <amaged__> #info Ahmed Maged
08:01:41 <morgan_orange> If future belongs to those who wake up early (french quote), it probably belong to Finland :)
08:01:53 <viktor_nokia> :)
08:02:21 <raghavendrachari> :)
08:02:21 <morgan_orange> #info agenda for today https://wiki.opnfv.org/functest_meeting
08:02:48 <morgan_orange> until the others join we may review the action point of last year...http://ircbot.wl.linuxfoundation.org/meetings/opnfv-testperf/2015/opnfv-testperf.2015-12-15-08.04.html
08:03:49 <morgan_orange> David is back from vacations today, he will check the status of the Orange POD, but theoretically it should be OK this week including SSH access to raghavendrachari
08:04:09 <morgan_orange> #info Orange POD status to be confirmed this week, ssh creds pushed to jumphost
08:04:25 <morgan_orange> #info Sprint 4 created, and additional Jira created
08:04:35 <morgan_orange> so action points are cleaned
08:04:50 <morgan_orange> #topic B-Release
08:05:18 <morgan_orange> #link https://jira.opnfv.org/secure/RapidBoard.jspa?rapidView=59
08:05:59 <morgan_orange> I saw that Jose, Valentin, Viktor and Juha worked during Xmas break, any update on the tests
08:06:31 <viktor_nokia> good progress with Tempest
08:07:01 <juhak> I'm currently going through all Rally cases in testcases/VIM/OpenStack/CI/rally_cert/scenario and making some modifications necessary
08:07:14 <juhak> there still are failures in some cases when several iterations are executed concurrently, will take a closer look on those next
08:07:43 <morgan_orange> #info good progress on tempest => we now use our custom suite https://build.opnfv.org/ci/view/functest/job/functest-fuel-opnfv-jump-2-daily-master/lastSuccessfulBuild/consoleText
08:08:02 <morgan_orange> #info progress in Rally with rally_cert scenario
08:08:26 <morgan_orange> Valentin_Orange: I was about to speak about vIMS but please go ahead...
08:09:29 <morgan_orange> #info vIMS updated => last run on Fuel/NoSDN OK https://build.opnfv.org/ci/view/functest/job/functest-fuel-opnfv-jump-2-daily-master/lastSuccessfulBuild/consoleText
08:10:31 <morgan_orange> so on the internal use case, still some open questions on ODL and ONOS (but dedicated topic later)
08:10:37 <morgan_orange> vPing OK
08:10:59 <morgan_orange> #info Jose also upgraded the system to clean the system after functest
08:11:18 <morgan_orange> #info regarding Functest versus Installer in Jenkins, the Status is today:
08:12:51 <morgan_orange> #info fuel/no SDN OK, compass partly OK (e.g. vIMS successfully tested manually but not through enkins, May-meimei listed prerequisites for Functests - access to admin network from Jumphost, ext network, ..), Apex?, Joid no progress - able to run partially on Orange POD but not yet on target lab (Intel Pod 5 due to creds retrieval issue)
08:13:01 <morgan_orange> #action morgan_orange fix joid creds issues asap
08:13:19 <morgan_orange> May-meimei: you want to add something regarding the work Functest/compass?
08:13:47 <May-meimei> we run odl test successful manually
08:14:09 <morgan_orange> #info odl tests successfully run manually on compass
08:14:24 <morgan_orange> so almost all the testcases have been successfully run manually on compass, right?
08:14:45 <May-meimei> most tests in tempest is failure
08:15:05 <morgan_orange> compass which is today the installer which looks the "most close" to the target SUT with 2 possible controllers ODL and ONOS
08:15:13 <lixiaoguang> #info lixiaoguang
08:15:26 <lixiaoguang> hello everyone, sorry for late
08:15:35 <morgan_orange> for Tempest did you run it recentently, I mean with the custom config file?
08:15:58 <morgan_orange> could be interesting to see the errors and discuss them with viktor_nokia
08:16:34 <May-meimei> ok, I have triggered in jenkins
08:16:41 <May-meimei> right now
08:16:48 <morgan_orange> #action May-meimei run tempest (recent version based on last rally version and custom config file) and troubleshoot with viktor_nokia
08:16:57 <morgan_orange> ok action points in real time then?
08:17:00 <leo_wang> how about the dashboard?
08:17:22 <leo_wang> compass does not present in it
08:17:42 <morgan_orange> you mean the test dashboard, right?
08:17:47 <leo_wang> yes
08:18:26 <morgan_orange> no installer is present yet, as LF web team prototype does nto include the filter per installer yet, but it was discussed in November and it is planned to have these filters for B-Release
08:18:40 <morgan_orange> basically today there are tests run on compass in the DB
08:19:30 <leo_wang> so finally we need to show results on board?
08:19:38 <morgan_orange> http://213.77.62.197/results?case=vIMS&installer=compass
08:19:49 <morgan_orange> http://213.77.62.197/results?case=vPing&installer=compass
08:20:01 <morgan_orange> these results must be consummed by LF for the dashboard
08:20:17 <morgan_orange> and the filter shall be able to compare the results
08:20:58 <morgan_orange> today the results are in the DB, the API allos LF to consume the results, LF made a first prototype (showed during the summit) we are waiting for an update of the dashboard including teh filter capability
08:21:17 <morgan_orange> but technically speaking, you have nothing to do, the results are properly stored in the DB...
08:21:25 <morgan_orange> so wait and see LF dashboard update
08:21:29 <leo_wang> :)
08:21:36 <morgan_orange> I sent a mail to Andrew and Marion yesterday
08:22:11 <chenshuai> did LF guys give the time line for this?
08:22:19 <morgan_orange> not yet
08:22:32 <chenshuai> sorry for this
08:22:59 <morgan_orange> but theoretically today is milestone E...so code freeze...
08:23:37 <morgan_orange> anyway congratulations to compass team, stil some work on automation to see all the tests executed automatically, but as most of the tests are running manually it should not be a big deal
08:23:40 <chenshuai> but bugfix should be accepted ? right?
08:23:54 <chenshuai> ok
08:23:54 <morgan_orange> yes of course
08:24:48 <morgan_orange> and I will be surprised that we really freeze assuming that today on jenkins (which is the reference for me) we are far from having the SUT ready with the different config, but will will see tonight during the release meeting
08:24:52 <chenshuai> morgan_orange: what about the community's strategy for opencontrail?
08:25:49 <morgan_orange> I do not really know....what I know is that ovno (opencontrail) contributed, they have scripts for the testing but today there is no target lab offerening a SUT including contrail
08:26:14 <morgan_orange> it could be probably not too complex with joid, as the juju charm exists but so far I do not see any lab with that config
08:26:31 <morgan_orange> so back to our Jira
08:26:41 <morgan_orange> #link https://jira.opnfv.org/secure/RapidBoard.jspa
08:26:57 <morgan_orange> we will have probably to priorize next week...
08:27:06 <chenshuai> morgan_orange: ok, has fuel finished integration with opencontrail? I am doing it for compass, but it's really hard way
08:27:15 <morgan_orange> no idea
08:27:27 <chenshuai> morgan_orange: all right
08:27:38 <morgan_orange> maybe a better view during Debra's meeting
08:28:05 <morgan_orange> #topic Functest E milestone
08:28:19 <morgan_orange> #info I started to complete the form for E milestone
08:28:27 <morgan_orange> #link https://wiki.opnfv.org/releases/brahmaputra/release_plan/functest_milestone_e_report
08:28:52 <morgan_orange> as far as I can see, it is still a pending actions for lots of projects
08:29:04 <morgan_orange> #action all review the wiki page on the E milestone
08:30:44 <morgan_orange> #info as mentioned in the page I have several concerns: not sure we can really extend internal ODL test suite as we imagined (we will discuss it in next topic on controller),  not sure SUT will be ready in time to allow a real troubleshooting campaign to ensure a minimum set of test consistency towards all the installers, and difficult to have a good view on the integration of our companion feature projects (as
08:31:41 <morgan_orange> for the companion projects, some were linked to odl test extension PolicyTest, some to additional tempest suites (SDNVPN) => this one we can achieve through the custom Tempest file
08:31:54 <morgan_orange> for Doctor, Promise and Copper, I do not know
08:32:24 <morgan_orange> r-mibu: you surely have a better view on the integration of doctor tests into CI for B release
08:34:01 <chenshuai> morgan_orange: SDN VPN for odl need use Opendaylight Beryllium release, I checked it on odl official web, it's still not released
08:34:18 <morgan_orange> for the implementation based on ODL
08:34:21 <chenshuai> https://jira.opnfv.org/browse/GENESIS-78
08:34:28 <morgan_orange> but there is also an implementation based on neutron extension (bagpipe)
08:34:34 <morgan_orange> this one does not require berylium
08:35:20 <chenshuai> do all the installers need to support this?
08:35:21 <morgan_orange> but you are right for the odl implementation the dependency towards berylium would probably prevent to get the feature in B first release
08:35:29 <chenshuai> or just an opption?
08:36:03 <morgan_orange> I am not directly involved neither in the feature(SDNVPN) nor installer so it is difficult for me to answer, I assume it is an option
08:36:22 <morgan_orange> all the feature project cannot ask all the installers to support everything...
08:36:55 <morgan_orange> it should be listed somewhere here https://wiki.opnfv.org/brahmaputra_testing_page
08:37:41 <morgan_orange> I think neutron BGPVPN extension is included in Liberty
08:37:49 <morgan_orange> but to be checked with SDNVPN team
08:38:00 <morgan_orange> or ask timirnich
08:38:46 <morgan_orange> #action morgan_orange task priorization for E release (but wait for meeting today)
08:38:46 <r-mibu> morgan_orange: I'm still trying to figure how we can create ci for feature projects. in doctor, I've created test scripts that can be run on vanilla opnfv.
08:39:21 <r-mibu> s/figure/figure out/
08:39:40 <r-mibu> https://gerrit.opnfv.org/gerrit/#/c/5439/
08:39:51 <morgan_orange> if you have scripts, what I can suggest...is to use Functest mechanism, we could add doctor script as part of the tests we run...but instead of calling our script we call yours
08:40:14 <morgan_orange> from the Functest docker, we clone several repo to get the scripts we want to run towards a SUT
08:40:43 <morgan_orange> we imagined a mechanism also to be sure we run only the tests taht make sense (not run ONOS suite towards OPNFV with ODL)
08:41:05 <morgan_orange> then in CI we have a scripts call run-test.sh where we can call the different testcase we want to run
08:41:22 <morgan_orange> we just launch scripts (vPing, vIMS, odl, ...) so why not doctor
08:41:49 <morgan_orange> the scripts include a setup and tear down section
08:42:37 <morgan_orange> so what I had in mind - it is the same for Onos on any feature tests is to run feature test suites from CI through the daily job run towards a fresh install
08:43:15 <morgan_orange> we have 2 types of jobs per installer  => one that runs all the tests one after the other, and one where you can run a single testcase
08:43:41 <morgan_orange> functest on compass dayly job => https://build.opnfv.org/ci/view/functest/job/functest-compass-huawei-us-deploy-bare-1-daily-master/
08:44:05 <morgan_orange> functest on compass with customization of testcase => https://build.opnfv.org/ci/view/functest/job/functest-compass-huawei-us-deploy-bare-1-suite-master/
08:44:27 <morgan_orange> r-mibu: if you want we can just try this option...as you want
08:44:39 <morgan_orange> #topic controller
08:45:02 <morgan_orange> lixiaoguang: pbandzi amaged__ raghavendrachari any update?
08:45:53 <pbandzi> #info i am trying to make odl test work also with mininet now. and in the future we can thonk how to get rod of moninet
08:46:23 <raghavendrachari> i workign on mininet docker setup
08:47:08 <lixiaoguang> #info The ONOSFW L3 test scripts and CI are almost finished
08:47:32 <raghavendrachari> pbandzi : shall i continue mininet installation
08:47:44 <pbandzi> raghavendrachari, can we have a chat after meeting to synchronize?
08:47:50 <raghavendrachari> ok
08:47:53 <pbandzi> raghavendrachari, yes please
08:48:36 <morgan_orange> ok for odl let's synchronize, code freeze is not planned for today for test projects..so we have additional time, do your best :), any visibility on the time needed for this refactoring?
08:49:43 <morgan_orange> for onos, lixiaoguang, as for Doctor, I suggest to add onos tests in the Functest test suites
08:49:55 <morgan_orange> Do we have a lab where we could test it at the moment?
08:50:44 <lixiaoguang> I'm now testing them in temp environment
08:50:54 <morgan_orange> chenshuai: May-meimei could the Huawei labs be configured to deploy alternatively Onos and ODL?
08:51:46 <May-meimei> virtual-deployment for this two both ok
08:52:30 <morgan_orange> ok so we probably could try on this config first before getting an alternance on bare metal target lab
08:52:43 <May-meimei> https://build.opnfv.org/ci/view/compass4nfv/
08:52:44 <lixiaoguang> I have tested the scripts on virtual-deployment
08:52:51 <lixiaoguang> it's ok
08:52:54 <morgan_orange> #action lixiaoguang May-meimei morgan_orange setup onos test from functest suite on compass virtual env
08:53:30 <morgan_orange> lixiaoguang: ok so it is just an integration into functest to be planned (just running the onos tests from Functest)
08:53:44 <pbandzi> morgan_orange: for ODL i think by the end of this week we can have something which will work
08:54:04 <lixiaoguang> yes
08:54:21 <morgan_orange> #action pbandzi raghavendrachari amaged__ synchro on work on ODL - target first refactoring before ennd of the week
08:54:27 <morgan_orange> pbandzi: ok great
08:54:34 <raghavendrachari> ok sure
08:54:59 <morgan_orange> as mentioned for the last one opencontrail, we got demo, apparently tests scripts are ready
08:55:36 <morgan_orange> but no target lab yet, I know that Juniper is setting an OPNFV Lab in new Jersey...it will probably short regarding the time constraint
08:56:05 <morgan_orange> easiest scenario as far as I can see is to deploy opencontrail with juju (as the integration already exists) but it has to be tested in the target labs
08:56:12 <morgan_orange> #topic AoB
08:56:15 <morgan_orange> 5 minutes left
08:56:41 <morgan_orange> for information I started the documentation (great update by r-mibu regarding the doc formats)
08:57:01 <morgan_orange> I planned to work on it this week (theoretically it should be finalized on E milestone...
08:57:20 <morgan_orange> another info, anyone from bottleneck project on the chan?
08:57:49 <morgan_orange> it seems that Jose's cleaning methods cleaned too much last night  http://pastebin.com/gqXdSJyb
08:58:19 <morgan_orange> any other topic to share, questions, concerns, remarks?
08:58:48 <viktor_nokia> no
08:58:58 <Valentin_Orange> no
08:59:09 <morgan_orange> ok so have a good busy week...
08:59:42 <morgan_orange> and may the force be with you...for E milestone
08:59:44 <morgan_orange> #endmeeting