08:00:59 <jose_lausuch> #startmeeting Functest weekly meeting April 12th 2016
08:00:59 <collabot> Meeting started Tue Apr 12 08:00:59 2016 UTC.  The chair is jose_lausuch. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:00:59 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:00:59 <collabot> The meeting name has been set to 'functest_weekly_meeting_april_12th_2016'
08:01:29 <jose_lausuch> #info agenda for today https://wiki.opnfv.org/display/functest/Functest+Meeting#FunctestMeeting-12/04(8UTC)
08:01:32 <jose_lausuch> #info Jose Lausuch
08:01:37 <jose_lausuch> pound in your names
08:01:39 <juhak> #info Juha Kosonen
08:01:47 <jose_lausuch> let's have2 min for people to join
08:01:56 <bertys___> #info Bertrand Souville
08:02:07 <lhinds> #info Luke Hinds
08:02:23 <JuhaHaapa> #info Juha Haapavirta
08:02:29 <CG_Nokia> #info CG_Nokia (Colum Gaynor)
08:03:31 <viktor_nokia> #info Viktor Tikkanen
08:03:51 <jose_lausuch> #topic Action point follow-up
08:04:17 <jose_lausuch> #info jose_lausuch: merge security epics and create appropriate task
08:04:35 <jose_lausuch> #info Done. I have created EPICs for each big feature/task we have in Functest. The idea is that everyone working on each epic create JIRA issues for smaller tasks and related to the corresponding epic. The JIRA shall be added to the commit message when applies. Everyone is free to create smaller tasks under an epics. It is the best way to keep track of the work being doing by each one of us.
08:04:56 <jose_lausuch> #link https://jira.opnfv.org/browse/FUNCTEST-183?jql=project%20%3D%20FUNCTEST%20AND%20issuetype%20%3D%20Epic
08:05:06 <ciprian-barbu> jose_lausuch: ok, go on with the meeting
08:05:08 <jose_lausuch> btw, just IRC, no gotomeeting
08:05:12 <jose_lausuch> :)
08:05:54 <jose_lausuch> we don't have more action points from last meeting, but every one has some task assigned
08:06:24 <jose_lausuch> we start to get some clearance from which feature projects will be integrated in functest
08:06:29 <jose_lausuch> but we can expect more
08:06:34 <jose_lausuch> #topic Feature project integration
08:06:41 <jose_lausuch> #info currently we have the following requests from feature projects:
08:06:52 <jose_lausuch> #info OVNO(OpenContrail) https://jira.opnfv.org/browse/FUNCTEST-183
08:07:01 <jose_lausuch> #info ONOS https://jira.opnfv.org/browse/FUNCTEST-182
08:07:06 <jose_lausuch> #info SDNVPN https://jira.opnfv.org/browse/FUNCTEST-181
08:07:10 <jose_lausuch> #info Security https://jira.opnfv.org/browse/FUNCTEST-169
08:07:14 <jose_lausuch> #info Congress https://jira.opnfv.org/browse/FUNCTEST-189
08:07:17 <jose_lausuch> #info Promise https://jira.opnfv.org/browse/FUNCTEST-162
08:07:20 <jose_lausuch> #info Doctor https://jira.opnfv.org/browse/FUNCTEST-192
08:07:29 <jose_lausuch> some of this have to be changed to epics... I think
08:07:34 <jose_lausuch> do I miss anything?
08:08:16 <jose_lausuch> bertys___: we have Promise also for Colorado, since we are gonna have more test, right?
08:08:35 <bertys___> right, thanks
08:08:51 <jose_lausuch> I will convert that task to an epic
08:09:09 <jose_lausuch> and you can create JIRA tickets on top of that epic
08:10:05 <jose_lausuch> lhinds: do you want to have a GoToMeeting session some day to talk about what can we include with regards to security?
08:10:20 <lhinds> IRC is ok jose_lausuch
08:10:37 <lhinds> oh some day
08:10:43 <jose_lausuch> or today
08:10:48 <jose_lausuch> I think we'll have time enough
08:11:31 <jose_lausuch> as you wish, we can have a session in gotomeeting (not today) or you can start giving us some hints about what we can include
08:11:47 <lhinds> I don't have anything prepared at the moment, so I can have something to share next meeting
08:12:04 <lhinds> I am happy to answer questions and give update though
08:12:20 <jose_lausuch> ok, shall we include it for next week's agenda?
08:12:25 <lhinds> yes please
08:12:27 <jose_lausuch> ok
08:12:39 <jose_lausuch> ok
08:12:40 <jose_lausuch> moving on
08:12:42 <lhinds> I had a question too..
08:12:45 <jose_lausuch> sure
08:12:50 <jose_lausuch> shoot
08:13:00 <lhinds> do we have a rough idea when flash test will be available to test with?
08:13:14 <jose_lausuch> enikher started yesterday on that
08:13:17 <jose_lausuch> so, probably soon :)
08:13:46 <enikher> I did 90% of the work yesterday
08:13:53 <lhinds> ok that's good, as that is my main integration point that is pending
08:14:00 <enikher> Could be that I am ready today or tomorrow
08:14:08 <jose_lausuch> enikher: that's great news, thanks!
08:14:17 <lhinds> ok, much quicker then expected
08:14:35 <lhinds> enikher, have you got alignment with each of the installers?
08:14:50 <lhinds> I know for example Apex only allow heat_admin who can sudo
08:14:53 <jose_lausuch> I think we'll start with fuel support, right?
08:15:00 <jose_lausuch> and then we will see how to handle it with others
08:15:14 <enikher> yes at the moment there is fuel support only.
08:15:19 <lhinds> ahh
08:15:25 <enikher> I don't have an apex deployment till now
08:15:33 <jose_lausuch> the problem with flashtest is that it's installer dependent
08:15:34 <lhinds> I still need to check the status oi
08:15:38 <enikher> I am setting up one at the moment but I need more time
08:15:42 <lhinds> of security scanning ubuntu nodes
08:15:49 <enikher> but the integration should not need longer than a day
08:15:50 <lhinds> they don't maintain scap very welll
08:15:50 <jose_lausuch> because we need to access the controllers from the installer' specific node
08:16:07 <lhinds> I can do a work around now for apex
08:16:10 <lhinds> no problem
08:16:25 <lhinds> one other quick one enikher
08:16:31 <lhinds> how will i connect?
08:16:43 <lhinds> will it be with a ssh_key, root or some sudo user?
08:16:51 <enikher> ssh_key
08:17:02 <enikher> in flash_test.yaml
08:17:10 <lhinds> perfect
08:17:16 <enikher> you can define connection to the main_controller
08:17:25 <enikher> through a instack vm or fuel
08:17:27 <enikher> as you like
08:17:45 <enikher> and then from this main_controller you can reach the other vms
08:17:50 <enikher> sorry not vms
08:17:54 <enikher> other computes
08:18:01 <lhinds> Would that be as a hop?
08:18:09 <lhinds> (other computes)
08:18:09 <enikher> yes
08:18:27 <enikher> you can define hops very easy. That is the basement for all
08:18:31 <lhinds> that might be tricky, I am doing this programatically with paramiko
08:18:45 <enikher> fuel->main_controller->compute->vm_23r23poj23423423423
08:19:26 <jose_lausuch> ok
08:19:28 <lhinds> ok, we can dig into that once you have your basic code in place
08:19:46 <lhinds> thanks enikher
08:19:51 <jose_lausuch> #info flash test probably completed this week, and lhinds can leverage it for the security test proposal
08:20:01 <jose_lausuch> thanks guys
08:20:40 <jose_lausuch> let's talk about tempest
08:20:49 <jose_lausuch> #topic Tempest way forward
08:20:51 <jose_lausuch> #link https://jira.opnfv.org/browse/FUNCTEST-172
08:20:59 <jose_lausuch> viktor_nokia: do you want to share your view?
08:21:29 <jose_lausuch> or your research ?
08:21:32 <viktor_nokia> #info measured smoke duration for apex: ~25min
08:21:57 <viktor_nokia> no other news...
08:22:23 <viktor_nokia> maybe we can exclude scenario test cases
08:22:34 <viktor_nokia> this will save 10-12 min
08:22:36 <jose_lausuch> viktor_nokia: can you update a new patch to have the execution of tempest/smoke and tempest/full ?
08:22:40 <jose_lausuch> removing the hardcoded list?
08:22:56 <jose_lausuch> enikher had the idea of posting a blacklist of test cases that we don't want to execute
08:23:00 <viktor_nokia> Yes, I will do it soon
08:23:40 <viktor_nokia> blacklist idea is good, I was thinking of it as well
08:23:48 <jose_lausuch> if you like, for each subtask you can define a new JIRA to keep track
08:23:51 <viktor_nokia> it can even be upstreamed to rally
08:23:52 <jose_lausuch> yes, I think so too
08:24:11 <jose_lausuch> instead of hardcoding the list of tests we execute, we leave that "intelligence" to tempest discover
08:24:18 <jose_lausuch> but with the possibility to exclude stuff
08:24:25 <viktor_nokia> yes
08:24:35 <jose_lausuch> yes, we'll ask rally community how we can help there
08:24:41 <jose_lausuch> maybe that's a good point to discuss
08:24:58 <viktor_nokia> OK. I will create needed subtasks...
08:25:05 <jose_lausuch> #info disucss with Rally community the possibility to add a blacklist for Rally verify (Tempest)
08:25:08 <jose_lausuch> thanks viktor_nokia
08:25:24 <jose_lausuch> the thing is that we need a basic smoke
08:25:34 <jose_lausuch> mainly in serial
08:25:43 <jose_lausuch> what ideas do you have for the full suite?
08:25:52 <jose_lausuch> can you info what happens when we run "full" ?
08:26:01 <jose_lausuch> difference between scenarios/normal tests
08:26:29 <viktor_nokia> We can run it to check how different installers are "compatible" with the upstream OpenStack
08:27:52 <viktor_nokia> #info measured full duration for apex: ~3,5 hours
08:27:52 <jose_lausuch> ok
08:27:59 <jose_lausuch> serial or parallel?
08:28:14 <viktor_nokia> I don't remember :(
08:28:21 <jose_lausuch> thats' way too much I think...
08:28:25 <viktor_nokia> I will check
08:28:26 <jose_lausuch> is that including scenarios as well?
08:28:32 <viktor_nokia> yes
08:28:41 <viktor_nokia> and many timeouts
08:28:47 <viktor_nokia> min each
08:28:51 <viktor_nokia> 5 min each
08:28:51 <jose_lausuch> I think we should tweak the timeouts as well
08:28:57 <jose_lausuch> and decrease them
08:29:07 <viktor_nokia> som work to do...
08:29:18 <jose_lausuch> but I propose to run it in parallel, so that we are also testing a "real life" case
08:29:34 <jose_lausuch> orchestrators/VNF managers might do things in parallel
08:30:13 <jose_lausuch> viktor_nokia: ok, please involve enikher in your ideas/thoughts. He has also some experience with tempest
08:30:24 <jose_lausuch> thanks
08:30:32 <viktor_nokia> BTW, it was parallel execution
08:30:53 <jose_lausuch> ok
08:31:01 <jose_lausuch> 3 hours is too much I imagine :)
08:31:07 <jose_lausuch> btw
08:31:13 <jose_lausuch> viktor_nokia: what is the status of nokia pod?
08:31:46 <viktor_nokia> we have some problem with internet connectivity from docker container
08:31:59 <viktor_nokia> https://build.opnfv.org/ci/job/functest-apex-nokia-pod1-daily-master/8/console
08:32:44 <viktor_nokia> ...correction regarding 'full' set: it was serial execution, not parallel...
08:33:14 <jose_lausuch> viktor_nokia:  ok
08:33:20 <viktor_nokia> we are working on that
08:33:21 <jose_lausuch> viktor_nokia: is that something we can fix?
08:33:33 <jose_lausuch> we can try an offline functest
08:33:42 <jose_lausuch> but... it hasnt been tested properly :)
08:34:15 <jose_lausuch> viktor_nokia: let's take it offline
08:34:17 <jose_lausuch> thanks
08:34:21 <jose_lausuch> at least some progress!
08:34:30 <viktor_nokia> It would be interesting to check if Functest can work offline
08:34:47 <jose_lausuch> yes, I think that should be a possibility for brahmaputra
08:34:51 <jose_lausuch> we tried once
08:34:55 <jose_lausuch> but there were too many dependencies
08:35:00 <jose_lausuch> but I think we can achieve it
08:35:07 <jose_lausuch> if we have everything pre installed
08:35:22 <viktor_nokia> OK
08:35:27 <jose_lausuch> ok
08:35:29 <jose_lausuch> next topic
08:35:34 <jose_lausuch> thanks viktor_nokia
08:35:36 <jose_lausuch> #topic Rally way forward
08:35:39 <jose_lausuch> #link https://jira.opnfv.org/browse/FUNCTEST-173
08:35:44 <jose_lausuch> juhak: any updates?
08:36:19 <juhak> executed tests using --smoke option, one heat failure, two nova failures
08:36:42 <juhak> total time ~37 min, nova itself took 24 min
08:37:21 <jose_lausuch> Juhak how many threads has that smoke job?
08:37:23 <juhak> so someting needs to be excluded in order reach 10 min goal
08:38:05 <juhak> smoke executed only one iteration per each test
08:38:08 <jose_lausuch> threads and parallel operations?
08:38:13 <jose_lausuch> and it takes 37 min?
08:38:14 <jose_lausuch> mmm
08:38:51 <jose_lausuch> can we have the basic openstack services only for the smoke?
08:38:52 <juhak> rally tests are executed serial, one after another
08:38:53 <jose_lausuch> so, no heat
08:39:00 <jose_lausuch> only nova/neutron... glance
08:39:38 <juhak> at the moment nova takes the longest time, then cinder (6mins)
08:40:10 <jose_lausuch> how many tests are executed in cinder?
08:40:20 <jose_lausuch> maybe we can set only 1 o 2 tests for cinder
08:40:23 <juhak> neutron 2.5 min
08:40:58 <juhak> yes, most like there is need to exclude some of cinder and nova tests
08:41:05 <jose_lausuch> ok
08:41:21 <jose_lausuch> I mean, we can have them always later on, for the full execution
08:41:28 <jose_lausuch> for the smoke, we should keep it simple
08:41:48 <juhak> yes, excluding them only in smoke case
08:42:51 <jose_lausuch> ok
08:42:57 <juhak> I'll for some further tests to get more accurate results
08:43:15 <jose_lausuch> juhak: but does it make sense at all to have a "smoke" rally?
08:43:30 <jose_lausuch> what do you think?
08:44:08 <juhak> it's useful at least when developing tests,
08:44:48 <juhak> a 'quick' way to see it basic things are working in all scenarios
08:44:54 <jose_lausuch> ok
08:45:07 <jose_lausuch> let's have it then
08:45:37 <jose_lausuch> juhak: anything else you want to share regarding rally?
08:45:51 <juhak> not at the moment
08:46:25 <jose_lausuch> #info Rally work ongoing, time measurements being performed to keep a simple rally-smoke test lasting max10 min. Some test have to be excluded
08:46:51 <juhak> how about the meeting with Rally guys, any news?
08:46:53 <jose_lausuch> #info rally-smoke as a 'quick' way to see it basic things are working in all scenarios
08:46:56 <jose_lausuch> yep
08:46:58 <jose_lausuch> nex topic
08:47:04 <jose_lausuch> #topic Rally sync meeting date/time
08:47:13 <jose_lausuch> #info it was planned for last week, but didn't get answer yet...
08:47:17 <jose_lausuch> I created this poll
08:47:23 <jose_lausuch> #info New time proposals: https://framadate.org/dYsCXViGCwVNqiZImuCAIdQy/admin
08:47:38 <jose_lausuch> please, for those who want to attend/participate, fill in your names and availability
08:47:51 <jose_lausuch> I'll try to contact boris-42 again
08:48:50 <jose_lausuch> the problem is the time difference
08:48:55 <jose_lausuch> let's see if we can handle that
08:49:20 <jose_lausuch> note the time is in UTC format
08:49:25 <jose_lausuch> to avoid confussion
08:50:41 <jose_lausuch> ok
08:50:47 <jose_lausuch> #topic AoB
08:50:54 <jose_lausuch> anyone wants to share something ?
08:52:44 <jose_lausuch> it seems not
08:52:54 <jose_lausuch> does anyone needs help with some tasks?
08:53:16 <serena-zte> Yes, I am
08:53:24 <jose_lausuch> serena-zte: sure, go ahead
08:53:29 <serena-zte> the new functest-178
08:53:54 <jose_lausuch> yes
08:54:15 <serena-zte> it evolves from functest-155 and functest-156
08:54:27 <jose_lausuch> yes, it's the "epic"
08:54:43 <jose_lausuch> but the "real" task is 155 and 156
08:55:24 <serena-zte> 155, I don't know what exactly I need to do
08:55:44 <jose_lausuch> ok, let's take it after the meeting
08:55:49 <jose_lausuch> I'll try to explain:)
08:55:56 <serena-zte> ok, thank you
08:56:04 <CG_Nokia> FUNCTEST-177: (ODL Suite Evolution) Too soon to ask help! Basic OPNFV Processes/Tools studied. Now studying Robot Framework... prerequisite to understand existing ODL testcases!
08:56:56 <jose_lausuch> CG_Nokia: ok, do you have questions about anything related to robot/odl suite?
08:57:12 <CG_Nokia> No... will study first a bit.
08:57:20 <jose_lausuch> please align with raghavendrachari
08:57:20 <jose_lausuch> ok
08:57:27 <CG_Nokia> okeydokey
08:57:47 <raghavendrachari> am studyin gsome docs for odl test suite evolution , derived some test cases , i'll update in jira
08:57:51 <jose_lausuch> feel free to drop an email with questions or just here on IRC
08:58:01 <jose_lausuch> raghavendrachari: ok, thanks
08:58:33 <jose_lausuch> anything else, guys ?
08:58:44 <raghavendrachari> yes
08:59:03 <jose_lausuch> raghavendrachari: go ahead
08:59:07 <raghavendrachari> apex three odl test cases has failed
08:59:14 <jose_lausuch> yes
09:00:07 <JuhaHaapa> i know the reason in our system: the odl controller is not running in the active openstack node
09:00:17 <raghavendrachari> i gone through logs but nothing found and in odl bugs i found some bug has raised regarding this errors as rase around c ondition
09:00:31 <JuhaHaapa> don't know how to fix it
09:00:47 <raghavendrachari> https://bugs.opendaylight.org/show_bug.cgi?id=3052
09:01:15 <jose_lausuch> mmmm
09:01:42 <raghavendrachari> i'll try to find out issue mostly one day or two days
09:02:00 <jose_lausuch> raghavendrachari: ok, that will be interesting to know why it fails
09:02:05 <jose_lausuch> it works on other installers
09:02:09 <jose_lausuch> on apex it fails always
09:02:11 <JuhaHaapa> ...of cource your errors could be different from mine raghavendrachari
09:02:14 <raghavendrachari> yeah..
09:02:17 <jose_lausuch> maybe due to what JuhaHaapa said
09:02:40 <jose_lausuch> feel free to discuss it on this channel guys
09:02:48 <jose_lausuch> we can also contact Apex guys
09:02:48 <raghavendrachari> ok i'll try to figure out ..
09:02:51 <jose_lausuch> to trobuleshoot this
09:02:57 <jose_lausuch> I have access to LF-pod1
09:03:05 <jose_lausuch> in case we need to try out something
09:03:07 <jose_lausuch> I can help
09:03:18 <jose_lausuch> ok, we need to close
09:03:25 <jose_lausuch> thanks all!
09:03:28 <jose_lausuch> #endmeeting