08:00:34 <morgan_orange> #startmeeting functest weekly meeting December 8th
08:00:34 <collabot> Meeting started Tue Dec  8 08:00:34 2015 UTC.  The chair is morgan_orange. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:00:34 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:00:34 <collabot> The meeting name has been set to 'functest_weekly_meeting_december_8th'
08:00:47 <morgan_orange> good morning / evening / night everybody
08:00:59 <morgan_orange> #info Morgan Richomme
08:01:05 <lixiaoguang> good afternoon
08:01:08 <lixiaoguang> ~~~
08:01:09 <viktor_nokia> #info Viktor Tikkanen
08:01:09 <pbandzi> #info Peter Bandzi
08:01:13 <juhak> #info Juha Kosonen
08:01:31 <amaged__> #info Ahmed Maged
08:01:32 <lixiaoguang> #info Xiaoguang
08:01:35 <morgan_orange> #topic agenda bashing
08:01:42 <morgan_orange> #link https://wiki.opnfv.org/functest_meeting
08:01:51 <morgan_orange> let's review action poitns of last week
08:02:10 <morgan_orange> #info all the feature projects have been contacted
08:02:23 <morgan_orange> #info DB ok for milestone D => 16 projects declared for 89 testcases
08:02:38 <morgan_orange> #undo
08:02:38 <collabot> Removing item from minutes: <MeetBot.ircmeeting.items.Info object at 0x2fd6750>
08:02:54 <morgan_orange> #info DB of for milestone D => 17 projects / 89 testcases
08:02:58 <morgan_orange> #link http://213.77.62.197/test_projects
08:03:44 <morgan_orange> #info regarding functest: companion projects (tests we may run from Functest in CI) are: ovno (Opencontrail), Doctor, Promise, PolicyTest (GBP in ODL) and part of SDNVPn (ODL part)
08:03:53 <morgan_orange> #info and Copper
08:04:02 <morgan_orange> #info + of course our own testcase..
08:04:36 <morgan_orange> #info second point OK  May-meimei is with her, she will give an overview of Compass/Functest work
08:05:15 <morgan_orange> #info point 3: list of openstack services supported initied but not finished => once Functest will be connected in CI to all installers, the information will be automatically retrieved
08:05:47 <morgan_orange> #info point 4: creation of ONOS testcase => done it is in the DB as 2 functest testcases onos and onos-ovsdb
08:06:01 <morgan_orange> so that is ok for the action points...or almost
08:06:19 <morgan_orange> #topic B Release Sprint 3 status
08:06:50 <morgan_orange> #info several actions in //: deployment of Functest on target labs, dev of testcases, integration of feature projects, docs....
08:06:59 <raghavendrachari> Hi all good day .. !!!
08:07:06 <morgan_orange> hi raghavendrachari
08:07:19 <morgan_orange> #info deployment of Functest in Lab
08:07:32 <morgan_orange> May-meimei: could you share the status of functet work in Huawei lab
08:07:48 <raghavendrachari> Hi morgan
08:08:13 <May-meimei> morgan_orange: we are integrating the test-suites now
08:08:21 <May-meimei> and we should also run yardstick
08:08:22 <May-meimei> those all test-suites will be completed before Jan
08:08:31 <May-meimei> I am not familiar with vims, we should deploy vIMS and test it?
08:08:45 <morgan_orange> #info Functest integration started on Compass => target Jan
08:09:20 <morgan_orange> theoretically....vims you just run and it should work..it is part of the tests to show the ability to deploy a compelte VNF (~ 10 VMs)
08:09:40 <morgan_orange> on which lab do you do that, could you put the jenkins link?
08:10:37 <morgan_orange> I added recently the Huawei US as a possible target as I saw this lab connected to CI. But as far as I understand you are performing the tests on the Huawei lab in Changai?
08:10:38 <May-meimei> jenkins link?
08:10:41 <jose_lausuch> hi
08:10:45 <jose_lausuch> sorry for the late
08:10:48 <jose_lausuch> #info Jose Lausuch
08:10:55 <jose_lausuch> my usual irc client doesn't work
08:11:40 <morgan_orange> the functest jenkins jobs are declared here: https://build.opnfv.org/ci/view/functest/
08:11:57 <morgan_orange> I created one for compass based on the deploy jobs I saw https://build.opnfv.org/ci/view/functest/job/functest-compass-master/
08:12:00 <May-meimei> I submit this patch:
08:12:00 <May-meimei> https://gerrit.opnfv.org/gerrit/#/c/3453/
08:12:34 <May-meimei> I think I should re-submit now
08:12:57 <morgan_orange> ok we may have some conflicts, but it shouyld be solvable :)
08:12:58 <morgan_orange> https://git.opnfv.org/cgit/releng/tree/jjb/functest/functest.yml
08:13:17 <May-meimei> ok!
08:13:17 <morgan_orange> I added the lab on my side https://git.opnfv.org/cgit/releng/tree/jjb/functest/functest.yml
08:13:26 <morgan_orange> but fortunately it is the same...
08:13:26 <May-meimei> I will work on it these days
08:13:29 <jose_lausuch> May-meimei: I think it looks better now
08:13:47 <jose_lausuch> before we were duplicating a lot of code
08:13:49 <morgan_orange> OK do not hesitate to add us (for review) if it helps
08:14:05 <May-meimei> jose_lausuch: thanks for your review
08:14:14 <jose_lausuch> no prob
08:14:14 <May-meimei> lots of suggestions
08:14:19 <jose_lausuch> :)
08:14:23 <morgan_orange> #info patch submitted for integration of the Compass ref lab into Jenkins https://gerrit.opnfv.org/gerrit/#/c/3453/
08:14:58 <jose_lausuch> there is one last comment
08:15:04 <jose_lausuch> adding compass in the "installer" area
08:15:13 <jose_lausuch> then I think we are fine with it
08:15:24 <morgan_orange> so we may say that Compass is on track, if help needed for any testcases, let us now. Some refactoring is planned soon on odl, and light refactoring also on Rally and Tempest but is shall be OK
08:15:46 <May-meimei> yes
08:15:46 <morgan_orange> #info Compass is on track: goal run functest from Jenkins asap
08:15:49 <jose_lausuch> May-meimei: which controllers are supported?
08:15:55 <morgan_orange> odl and onos
08:15:58 <jose_lausuch> ok
08:16:11 <May-meimei> both
08:16:16 <morgan_orange> but I do not know if there is a deployment strategy for that?
08:16:31 <morgan_orange> I saw that both jenkins jobs were available
08:16:33 <jose_lausuch> deployment strategy?
08:16:39 <morgan_orange> #info onos and odl supported by compass
08:16:52 <morgan_orange> #link https://wiki.opnfv.org/brahmaputra_testing_page
08:17:11 <morgan_orange> jose_lausuch: do they plan to alternate installation with odl/onos
08:17:24 <morgan_orange> what kind of fresh install could we expect
08:18:00 <jose_lausuch> you mean compass?
08:18:04 <morgan_orange> yes
08:18:10 <jose_lausuch> dont ask me:D
08:18:22 <morgan_orange> yes it is a question for MatthewLi or fdegir
08:18:25 <May-meimei> what about other installers?
08:18:28 <jose_lausuch> I think it would be ideal
08:18:37 <jose_lausuch> to have alternate controllers every deployment
08:18:47 <morgan_orange> but it will be probably the first example on which we can work to run functest with different config
08:18:51 <jose_lausuch> I will ask for Fuel what the plan is
08:19:00 <jose_lausuch> yes
08:19:24 <morgan_orange> #info once we will get the lab ready, discuss the deployment strategy regarding the controllers and/or options (specific OVS, kvm, QEMU,...)
08:20:04 <morgan_orange> Ok so I think it is OK for compass, congratulations for the great work, with 4 existing jenkins jobs covering virtual/bare metal and 2 controlelrs onos/odl you are not late...
08:20:17 <morgan_orange> jose_lausuch: status for apex and fuel?
08:20:37 <jose_lausuch> we tried to fix the docker stuff for apex
08:20:53 <jose_lausuch> #info apex almost ready to run functest
08:21:02 <jose_lausuch> just some networking issues in the jumphost
08:21:05 <jose_lausuch> iptables rules
08:21:07 * fdegir working on creating different daily jobs that run deployments with different controllers
08:21:12 <morgan_orange> #info for the moemtn 1 combination apex/odl
08:21:17 <jose_lausuch> we couldnt ping the installer vm from the container
08:21:28 <jose_lausuch> fdegir: ok
08:21:37 <fdegir> if you can review this https://gerrit.opnfv.org/gerrit/#/c/4003/
08:21:56 <fdegir> then I know I'm going to the right direction
08:22:12 <jose_lausuch> #info for fuel, there are problems with ODL tests due to the default port for the rest api (we have to change it)
08:22:13 <morgan_orange> #action all review patch for deployment strategy https://gerrit.opnfv.org/gerrit/#/c/4003/
08:22:39 <morgan_orange> #info for joid, tests done from CI on orange pod (not the target pod, wait for connection of INTEL POD 5 to CI)
08:22:57 <morgan_orange> #info joid => able to run docker from CI, able to run vPing and tempest
08:23:31 <morgan_orange> #info joid => upgrade planned on orange pod 2 today (to move to Liberty and add Ceph, heat, ceilometer) to be closed to target release...
08:23:47 <morgan_orange> so juhak and viktor_nokia the lab may not be accessible during the upgrade
08:23:58 <morgan_orange> David_Orange:  you have some details and the timing?
08:24:15 <jose_lausuch> morgan_orange: is joid already supporting liberty?
08:24:25 <morgan_orange> yes
08:24:28 <David_Orange> yes
08:24:29 <jose_lausuch> great
08:24:38 <David_Orange> i plan to install it this afternoon
08:24:45 <morgan_orange> it will save de facto dependency issues for some projects
08:25:21 <morgan_orange> #info upgrade joid orange-pod2 planned this afternoon (not available for testing) => move to liberty + heat/ceilometer/ceph
08:25:31 <morgan_orange> OK I think that is ok for the view of the deployment
08:26:01 <morgan_orange> raghavendrachari: I will create an access to Orange Pod so we could work together on the platform
08:26:08 <viktor_nokia> cinder was missing as well?
08:26:48 <raghavendrachari> ok sure , i'll
08:27:04 <morgan_orange> yes, it was really an alpha version, next one would be close to target
08:27:10 <morgan_orange> #topic ODL
08:27:37 <morgan_orange> important internal use case, all the more as, feature projects "just" want to import upstream ODL Robot suite related to their feature
08:28:02 <morgan_orange> pbandzi: any update to share
08:28:06 <pbandzi> #info so far it looks like we can add tests from suite: https://github.com/opendaylight/integration/tree/master/test/csit/suites/ovsdb
08:28:31 <pbandzi> #info jira for those suit to be created today
08:29:04 <pbandzi> #info looks however bigger task that expoected :)
08:29:27 <morgan_orange> shall we put a warning on this for the release meeting?
08:30:07 <morgan_orange> you think it is possible to include the upstream suite for PolicyTest and SDNVPN?
08:30:20 <pbandzi> not need yet. we ';; try to make ti work this week
08:30:24 <jose_lausuch> sdnvpn will be added by Tim
08:30:36 <jose_lausuch> with some modifications
08:31:13 <pbandzi> i'll look for policy test also not yet checked
08:31:14 <morgan_orange> added in upstream or in functest, my assumption is that refactoring is neede to run the robot suites
08:31:19 <jose_lausuch> pbandzi: what about including mininet? https://github.com/opendaylight/integration/blob/732659be50eedc962e9cf09b09ac15e459523c7a/test/csit/libraries/MininetTopo/create_fullymesh.py
08:31:29 <jose_lausuch> is it worth it?
08:31:45 <pbandzi> i thnk for ovsdb it will not be needed
08:31:55 <jose_lausuch> ok
08:32:10 <pbandzi> if I found some test which are worth to add we can add mininet also , but so far we don't need it
08:32:20 <morgan_orange> pbandzi: you may ask for support from amaged__ (but I know that you are connected) or raghavendrachari (who knows also ODL)
08:32:36 <morgan_orange> do not hesitate to delegate
08:32:37 <morgan_orange> :)
08:32:38 <pbandzi> we are working together on it with amaged__
08:33:06 <morgan_orange> ok so let's see the progress this week
08:33:10 <amaged__> Deja :)
08:33:24 <morgan_orange> #topic Management of constraints in CI
08:33:53 <morgan_orange> so knwo we know what we have to test, we know where (at least for first steps)
08:34:15 <morgan_orange> we should put in place a mechanism to run the tests in CI accordingly to the fresh install we have in front of our tests
08:34:25 <morgan_orange> fdegir: initiates work on Ci regarding this
08:34:57 <morgan_orange> #info question on constarint management in Functest on CI
08:35:03 <morgan_orange> #info work in progress in releng
08:35:06 <morgan_orange> #link https://gerrit.opnfv.org/gerrit/#/c/4003/
08:35:28 <morgan_orange> the CI will provide as inputs information on the controller, options
08:35:44 <morgan_orange> then when running functest we will check that the testcase is just runnable
08:35:58 <jose_lausuch> yes
08:35:59 <morgan_orange> e.g. no need to try to run odl with a fresh install with onos...
08:36:21 <morgan_orange> or no need to run SDVPN if OVS is not > 2.4 (as far as i remember) because feature needed will not be available
08:36:33 <morgan_orange> My view was to create a file listing these dependencies
08:36:35 <raghavendrachari> yes , i worked on odl related stuff , i can do up to my level
08:36:39 <jose_lausuch> did you info your patch?
08:36:59 <morgan_orange> #info work on file listing dependencies prior to test
08:37:05 <jose_lausuch> #info test.yaml to list the config for each test project
08:37:05 <jose_lausuch> https://gerrit.opnfv.org/gerrit/#/c/4021/
08:37:08 <jose_lausuch> #info https://gerrit.opnfv.org/gerrit/#/c/4021/
08:37:08 <morgan_orange> #link https://gerrit.opnfv.org/gerrit/#/c/4021/
08:37:12 <jose_lausuch> sorry...
08:37:41 <morgan_orange> instead of running in the docker ./runtest -t rally, tempest,vping
08:37:43 <pbandzi> #info corection to ODL... we will also include mininet...
08:37:58 <jose_lausuch> oh!
08:37:59 <jose_lausuch> :)
08:38:08 <morgan_orange> we may reun something like ./run_test.sh -c opnfv-tests.yml
08:38:42 <morgan_orange> then we try by default to run all our testcases (internal + companion) but before we just check the jenkins inputs versus declared min config
08:38:50 <morgan_orange> does it make sense for everybody?
08:38:58 <jose_lausuch> +1
08:39:15 <morgan_orange> I had a discussion with fdegir this morning, anac1 do you agree
08:39:35 <morgan_orange> basically for the moment only 2 test projects have companion projects (where we need to get the info)
08:39:40 <morgan_orange> functest and yardstick...
08:40:01 <morgan_orange> the other test projects could directly deal with the inputs from jenkins as they just deal with their own testcases
08:40:10 <morgan_orange> so we coudl imagine to have separate files
08:40:15 <morgan_orange> but as we put everything in a DB
08:40:27 <morgan_orange> i think it is usefull to have only 1 config file
08:40:31 <morgan_orange> to be consistent...
08:40:34 <morgan_orange> what do you think?
08:40:45 <jose_lausuch> I agree in having 1 config file for all
08:41:13 <morgan_orange> #action jose_lausuch morgan_orange create function to deal with opnfv-test file and add support for -c in run_test
08:41:23 <morgan_orange> ok I will ask anac1 offline
08:41:40 <jose_lausuch> to be put in releng repo of course
08:41:44 <morgan_orange> yes
08:41:53 <morgan_orange> #topic start speaking of documentation....
08:41:58 <jose_lausuch> fdegir: the opnfv directory is getting full of things :)
08:42:11 <morgan_orange> #info ryota sent a mail expalining some changes in doc management
08:42:18 <morgan_orange> I will try to initiate something
08:42:30 <morgan_orange> for functest we need a user guide, an installation guide
08:42:34 <jose_lausuch> We need a lot of updates in our docs
08:42:48 <morgan_orange> is it an update or a new one...
08:42:51 <jose_lausuch> fortunatelly, it will be similar to what we have in the docker wiki
08:43:00 <jose_lausuch> maybe a new one? reusing some stuff?
08:43:06 <morgan_orange> yes but we need the description of the testcases
08:43:11 <jose_lausuch> sure
08:43:12 <morgan_orange> as we are moving to sphinx
08:43:20 <morgan_orange> I think we can say it will be a new one...
08:43:31 <morgan_orange> #action morgan_orange initiate functest doc
08:43:43 <morgan_orange> we will have also to produce a high level testcase view based on the DB
08:44:12 <morgan_orange> that is an activity post xmas break, but i will try to initiate something anyway
08:44:17 <morgan_orange> #topic AoB
08:44:27 <jose_lausuch> I would say, we do DOC stuff after the code freeze
08:44:27 <morgan_orange> maybe speak of Rally refactoring
08:44:37 <morgan_orange> #undo
08:44:37 <collabot> Removing item from minutes: <MeetBot.ircmeeting.items.Topic object at 0x322a590>
08:44:52 <morgan_orange> #info Doc after E milestone (beginning of january)
08:44:56 <morgan_orange> #topic AoB
08:45:12 <morgan_orange> juhak: maybe share info on the last Jira with the team
08:46:02 <juhak> I split the refactoring ticket into two new ones
08:46:30 <juhak> and start working with parameterizing Rally scenarios at first
08:46:52 <morgan_orange> #info Rally refactoring in progress => split into 2 JIRAs
08:46:55 <morgan_orange> #link https://jira.opnfv.org/browse/FUNCTEST-109
08:47:04 <morgan_orange> #link https://jira.opnfv.org/browse/FUNCTEST-110
08:47:09 <juhak> was there something related to Rally and vPing too?
08:47:31 <morgan_orange> only Rally
08:47:37 <juhak> ok
08:47:55 <morgan_orange> I suspend my work on rally 2 dashboard
08:48:04 <morgan_orange> as the results may changes in the DB after the refactoring
08:48:33 <jose_lausuch> juhak: is there a automatic way to detect a proper way of defining the (e.g.) number of iterations and concurrency?
08:48:35 <morgan_orange> wait what kind of artifact we will produce and push into the DB before writing any post processing
08:48:54 <jose_lausuch> or the users has to modify these values depending on the amount of HW they have
08:49:02 <juhak> need to study that
08:49:16 <jose_lausuch> I would like to imagine something like this:
08:49:40 <jose_lausuch> I am a user X and have a POD with 6 servers to install OPNFV --> recommended config A
08:49:51 <jose_lausuch> I am a user Y and have a POD with 20 servers to install OPNFV --> recommended config B
08:49:53 <jose_lausuch> and so on
08:49:58 <jose_lausuch> if that was automatic would be awesome
08:50:21 <juhak> yes it would
08:50:21 <jose_lausuch> because the user doesnt have a clue how many VMs to create with Rally concurrently
08:50:33 <morgan_orange> theoretically the Pharos target shall be almost the same...but for tests on Labs with additional capabilities it will be real audit tool
08:50:37 <jose_lausuch> maybe if Rally doesnt provide that yet, we can have some recommendations
08:50:53 <morgan_orange> BTW https://jira.opnfv.org/browse/FUNCTEST-71 closed
08:50:54 <jose_lausuch> or different config files
08:50:58 <jose_lausuch> like
08:51:05 <jose_lausuch> rally-3servers.yaml
08:51:17 <jose_lausuch> rally-6servers.yaml
08:51:22 <jose_lausuch> rally-20servers.yaml
08:51:23 <jose_lausuch> and so on
08:51:28 <jose_lausuch> like defaults
08:51:39 <morgan_orange> #info suggestion jose_lausuch find an automatic way to detect a proper way of defining the (e.g.) number of iterations and concurrency => propose some default configuration depending on the pod config
08:52:05 <morgan_orange> we may imagine a rally-pharos sticking to pharos definition (3 control nodes + 2 compute nodes)
08:52:18 <jose_lausuch> yes, I would say that is the default config
08:52:30 <jose_lausuch> and then , have other different templates for different number of servers
08:52:33 <jose_lausuch> (for example)
08:53:11 <morgan_orange> ok let's juhak set the scenario in place and let's discuss it next week
08:53:30 <juhak> sounds good to me
08:53:33 <morgan_orange> #info Tempest will be also a little bit refactored with the creation of our own suite (no more smoke)
08:54:03 <morgan_orange> hope the interruption on orange pod will not be too long :)
08:54:40 <morgan_orange> #info demo Juniper this week (postponed from last week), you can join
08:55:13 <morgan_orange> ovno team already shows some code...need an OPNFV labs running with Opencontrail to start testing it
08:56:14 <morgan_orange> for the Release meeting this afternoon, I think we shall be able to run Functest towards 3 or 4 installers before the end of the week (in their current config)
08:56:20 <morgan_orange> so this status is green
08:56:58 <morgan_orange> for the testcases, it is still green, we will see next week regarding odl if the work is not much bigger than expect but work is in progress
08:57:38 <morgan_orange> for the integration of feature tests, it is Orange as it is still difficult to have the visibility but all the projects are working hard..
08:58:01 <jose_lausuch> what do you mean?
08:58:05 <jose_lausuch> dont get that
08:58:41 <morgan_orange> regarding the timing, I am not able to say when we will be able to test ovno but I know that ovno already has some code written for their tests
08:58:48 <morgan_orange> and the same for other companion projects
08:58:54 <jose_lausuch> ok
08:59:44 <morgan_orange> #info For release meeting: 1) we shall be able to run Functest towards 3 or 4 installers before the end of the week
09:00:03 <morgan_orange> #info For release meeting: 2) testcases, it is still green, we will see next week regarding odl if the work is not much bigger than expect but work is in progress
09:00:22 <jose_lausuch> for 1)  from the CI
09:00:33 <morgan_orange> #undo
09:00:33 <collabot> Removing item from minutes: <MeetBot.ircmeeting.items.Info object at 0x322ad10>
09:00:38 <morgan_orange> #info for CI....
09:00:53 <morgan_orange> #info For release meeting: 2) testcases, it is still green, we will see next week regarding odl if the work is not much bigger than expect but work is in progress
09:00:54 <morgan_orange> #info for Release meeting 3) feature projects started showing code, but not full visibility on when we could be able to test
09:01:27 <morgan_orange> #action morgan_orange contact anac1 regarding the test config file in releng
09:01:34 <morgan_orange> any other stuff to share?
09:01:45 <jose_lausuch> no
09:01:49 <jose_lausuch> but something will come up :)
09:01:53 <morgan_orange> heavy week...
09:01:59 <morgan_orange> for everybody
09:02:09 <jose_lausuch> cheer up everyone, lets make this work!
09:02:18 <morgan_orange> I will not available last week but jose_lausuch will chair...
09:02:23 <morgan_orange> oups
09:02:25 <morgan_orange> already tired
09:02:27 <morgan_orange> next week
09:02:35 <jose_lausuch> ok
09:03:18 <morgan_orange> I may try (stupid team building meeting...at 9 we usually have not started)
09:03:24 <morgan_orange> ok enjoy your week
09:03:41 <jose_lausuch> thanks morgan_orange
09:03:43 <morgan_orange> #endmeeting