08:00:08 <morgan_orange> #startmeeting Functest weekly meeting August 30th
08:00:08 <collabot> Meeting started Tue Aug 30 08:00:08 2016 UTC.  The chair is morgan_orange. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:00:08 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:00:08 <collabot> The meeting name has been set to 'functest_weekly_meeting_august_30th'
08:00:13 <morgan_orange> #topic call role
08:00:19 <morgan_orange> #info Morgan Richomme
08:00:23 <jose_lausuch> #info Jose Lausuch
08:00:29 <raghu_> #info raghavendrachari
08:00:32 <juhak> #info Juha Kosonen
08:00:35 <viktor_t> #info Viktor Tikkanen
08:00:38 <boucherv_orange> #info Valentin Boucher
08:00:59 <morgan_orange> #agenda: https://wiki.opnfv.org/display/functest/Functest+Meeting
08:01:11 <morgan_orange> #topic Action point follow up
08:01:24 <morgan_orange> #info AP1: morgan_orange SerenaFeng sync with Juraj for dashboard activity
08:01:30 <morgan_orange> #action morgan_orange SerenaFeng sync with Juraj for dashboard activity
08:01:55 <morgan_orange> no work on this (focus on troubleshooting / scenario)
08:01:59 <morgan_orange> #info AP2: morgan_orange lhinds see why security_scan is no more triggered on apex scenario
08:02:05 <morgan_orange> #info done, scenario constraint modified, security_scan now run, see next sections for discussion on issues
08:02:13 <morgan_orange> #info AP3: ollivier May-meimei clarify port management on ODL
08:02:18 <morgan_orange> #info done
08:02:24 <morgan_orange> #info AP4: morgan_orange contact Apex to see why 50% of the scenarios are FAIl due to creds
08:02:32 <morgan_orange> #info done: scenario whith failed credentials correspond to failed apex deployement...they try to run functest even if the SUT is not properly deployed
08:02:40 <morgan_orange> #info AP5: ollivier look at https://build.opnfv.org/ci/view/functest/job/functest-apex-apex-daily-master-daily-master/296/console
08:02:47 <morgan_orange> #info error not reproduced https://build.opnfv.org/ci/view/functest/job/functest-apex-apex-daily-colorado-daily-colorado/35/console
08:02:53 <morgan_orange> #info AP6: morgan_orange clean JIRAs
08:03:01 <morgan_orange> #info done even if it is not completed, David's last mail showed lots of unassigned (JIRA for which we did not put the release version for the fix) + 11 / 50 issues assigned to Colorado 1.0 that are unresolved
08:03:13 <morgan_orange> #action morgan_richomme re-clean based on David's list
08:03:21 <morgan_orange> #info for Colorado Release manager said that now only bug fixes must be cherry picked and correspond to a JIRA
08:03:26 <morgan_orange> #action morgan_richomme create JIRAs if needed to reflect bug fix activity
08:03:33 <morgan_orange> #info AP7 morgan_orange lauch commiter promotion procedure for SerenaFeng and ollivier
08:03:39 <morgan_orange> #info done, wait for the last committer vote
08:03:55 <jose_lausuch> who is missing?
08:04:15 <morgan_orange> May-meimei: do you know the other Huawei committers in Functest (only their vote is missing)
08:04:55 <morgan_orange> Qinglong Lan, Li Xiaoguang and Zhang Haoyu
08:05:03 <JuhaHaapa> #info Juha Haapavirta
08:05:20 <May-meimei> morgan_orange: I know them
08:05:28 <morgan_orange> any question regarding the action points, we will discuss secruity scan, dashboard and scenarios in the next sections
08:05:31 <May-meimei> will notice them
08:05:42 <morgan_orange> May-meimei: ok thanks
08:06:01 <morgan_orange> #topic Colorado status
08:06:28 <morgan_orange> #info scenario status: apex, fuel and joid colorado run started
08:06:39 <morgan_orange> any news on compass colorado job?
08:07:29 <morgan_orange> dedicated reporting pages created: http://testresults.opnfv.org/reporting/functest/release/colorado/index-status-apex.html, http://testresults.opnfv.org/reporting/functest/release/colorado/index-status-compass.html, http://testresults.opnfv.org/reporting/functest/release/colorado/index-status-fuel.html, http://testresults.opnfv.org/reporting/functest/release/colorado/index-status-joid.html
08:07:39 <morgan_orange> master reporting still available
08:07:58 <morgan_orange> #info colorado reporting available
08:08:06 <CG_Nokia> #info CG_Nokia
08:08:09 <lhinds> lhinds is sorry for being late
08:08:29 <jose_lausuch> mmm if it wasnt for copper test we would have green in 2 scenarios for apex
08:08:49 <jose_lausuch> same for parser/fuel
08:08:57 <ollivier> #info ollivier
08:09:05 <morgan_orange> yep we will detail a little bit as we have already some answers
08:09:18 <morgan_orange> #info apex: copper => mail from Tim, a fix will be provided
08:09:33 <morgan_orange> #info apex without copper => 2 scenarios will be already validated
08:09:43 <morgan_orange> #info apex/security scan => I will create a Jira
08:10:08 <morgan_orange> lhinds did you have time to look at the python error
08:10:35 <lhinds> not seen that yet, it was public holiday yesterday..do you have a link to the failed job?
08:11:17 <morgan_orange> lhinds any colorado job, error is the same whatever the scenario
08:11:29 <morgan_orange> installation looks fine
08:11:47 <jose_lausuch> morgan_orange: this page doesnt work http://testresults.opnfv.org/reporting/functest/release/colorado/index-status-compass.html
08:12:12 <morgan_orange> #action morgan_orange check reporting links  http://testresults.opnfv.org/reporting/functest/release/colorado/index-status-compass.html
08:12:22 <morgan_orange> there is no compass job on colorado
08:12:32 <May-meimei> morgan_orange: compass job will begin to run soon
08:12:41 <morgan_orange> so jose_lausuch it is normal...
08:13:08 <morgan_orange> lhinds https://build.opnfv.org/ci/view/functest/job/functest-apex-apex-daily-colorado-daily-colorado/42/console
08:13:26 <morgan_orange> let's work on it offline
08:13:40 <morgan_orange> #info compass colorado jobs will be soon available
08:14:01 <lhinds> do you mean 'sudo: hiera: command not found'?
08:14:43 <jose_lausuch> ah ok
08:15:27 <lhinds> oh i see it
08:15:39 <morgan_orange> #info compass/moon: discussion yesterday by mail - still some errors: odl (first test / AD-SAL), Tempest errors (as joid/xenial scenario), Moon (internal hardcoded passwords)
08:15:53 <lhinds> ok, I will raise a jira and fix
08:16:25 <morgan_orange> #info I suggest for the odl test to do nothing, i.e. accept that the first test is failed (normal because moon, that does an identitiy federation dit it only partially for odl) and document it
08:17:32 <morgan_orange> #info fuel/parser issue => mainly on noha scenarios (problem of resources in nonha pod?) => shall we consider a constraint in parser declaration to run it only on ha scenario?
08:18:27 <jose_lausuch> we can ask them
08:18:31 <jose_lausuch> but it might be the reason
08:18:35 <jose_lausuch> it als fails in virtual deplyometns
08:19:08 <morgan_orange> SerenaFeng: is working with parser team, I do not know if we have a clear view on ressources in non ha...
08:19:54 <SerenaFeng> mainly on virtual pods
08:20:01 <morgan_orange> #action SerenaFeng see with parser team if parser must be run on HA mode only
08:20:03 <SerenaFeng> success on all physical pods
08:20:17 <SerenaFeng> not related to HA mode
08:20:55 <SerenaFeng> we think maybe resources on virtual env is not big enough
08:20:56 <morgan_orange> usually HA are on physicall modes, non HA on virtual, but it is not a rule
08:21:03 <SerenaFeng> I have reply the email
08:21:23 <SerenaFeng> all failure due to No host available
08:22:19 <SerenaFeng> we can discuss it offline
08:22:24 <morgan_orange> ok
08:22:51 <SerenaFeng> I will trouble shooting it with parser people asap
08:22:52 <morgan_orange> #info joid/domino: domino excluded from automatic reporting => manual workaround aggreed by domino and indicated in release note
08:23:21 <morgan_orange> #info joid/lxd scenario: discussion offline the cirrios lxd image does not display any log in the nova console
08:23:47 <morgan_orange> #info joid/lxd: suggestion remove tests relying on nova log for lxd scenarios and document it in the release note
08:23:50 <morgan_orange> any objection?
08:24:10 <jose_lausuch> no
08:24:14 <jose_lausuch> its what we can do for now
08:24:14 <morgan_orange> it means no healthcheck for lxd scenario as last test for DHCP is based on nova logs
08:24:33 <morgan_orange> and create a JIRA for improvement in healthcheck for D
08:24:42 <jose_lausuch> Im not 100% happy with that, but we can't do much about it now
08:24:54 <morgan_orange> #action morgan_orange adapt lxd scenario declaration + release note accordingly
08:24:57 <jose_lausuch> its a test problem, not the scenario functionality
08:25:32 <morgan_orange> #info all odl_l3 scenarios failed due to tempest not reaching 100%
08:25:50 <morgan_orange> #action viktor_t see if for odl_l3 we should exclude test cases
08:25:53 <viktor_t> I have one comment/question...
08:26:05 <viktor_t> I've checked this morning many recent functest.log files from different pods in artifacts but in all the logs tempest cases passed without a single error
08:26:16 <viktor_t> e.g. in http://artifacts.opnfv.org/logs_functest_huawei-pod1.html
08:26:30 <viktor_t> it seems that something has been changed between 27.7 and 12.8 so that after 12.8 only OK logs are stored into artifacts??
08:27:22 <viktor_t> was just seeking for odl_l3 related logs...
08:27:45 <morgan_orange> viktor_t: hmm, it corresponds to the change in jenkins, we exit -1 when there are errors and I assume du to this exit we do not call the function to push results to artifacts
08:27:59 <morgan_orange> we push when everything is OK...
08:28:11 <jose_lausuch> thats bad
08:28:13 <jose_lausuch> we should change this
08:28:21 <viktor_t> should it be in other way?
08:28:32 <viktor_t> we don't need OK logs :)
08:28:37 <jose_lausuch> +1
08:28:43 <juhak> +1
08:28:51 <morgan_orange> +1
08:28:53 <viktor_t> anyway, for me it looks like tempest issues in odl_l3 are not common to all the installers
08:29:02 <viktor_t> 1) failed test cases are not always the same in different pods
08:29:15 <viktor_t> 2) some test cases either fails or passes in consecutive runs (see e.g. os-odl_l3-nofeature-ha at http://testresults.opnfv.org/reporting/functest/release/master/index-tempest-compass.html
08:29:31 <morgan_orange> #info issues with artifact: we push only the results of succesful scenarios
08:29:44 <viktor_t> looks like ODL timing issue etc.
08:30:09 <morgan_orange> #info it is probably due to the refactoring of the functest job, if we exit -1, we do not push the results, the -1 should be only at the end of the run including results pushed to artifact
08:30:24 <morgan_orange> jose_lausuch: I can action you for that?
08:30:36 <jose_lausuch> morgan_orange: yes
08:30:39 <jose_lausuch> jira bug
08:31:02 <morgan_orange> #action jose_lausuch fix issue of artifacts not pushed in cased of exit -1 (any case failed)
08:31:48 <morgan_orange> #info regarding odl_l3 viktor_t says that errors are different depending on installers..but logs are needed...and as logs are not pushed when errors we do not have them. First fix artifact then troubleshoot odl_l3 scenarios
08:32:28 <morgan_orange> #action viktor_t once artifact issue fixed, study odl_l3 tempest errors
08:32:46 <morgan_orange> #info bgpvn troubleshooting in progress
08:32:56 <morgan_orange> that is all for the scenarios for me
08:33:11 <morgan_orange> #info bug/fix / framework
08:33:30 <morgan_orange> #info ollivier pushed some patches to fix the messy logs in jenkins
08:33:50 <morgan_orange> ollivier: is it ready to be merged?
08:33:59 <ollivier> morgan_orange: I think so
08:34:33 <morgan_orange> #link https://gerrit.opnfv.org/gerrit/#/c/19793/
08:34:57 <morgan_orange> OK I will do after the meeting
08:35:00 <ollivier> morgan_orange: we should check if performance decreases as we stop buffering. But I should be ok.
08:35:41 <morgan_orange> ollivier: ok thanks for the correction
08:35:48 <jose_lausuch> the ideal case is that we dont use "print" functions, and we use logger for everything
08:36:08 <jose_lausuch> who can take care of removing logger as a function parameteres in functest_utils?
08:36:10 <morgan_orange> jose_lausuch: yes but we have a heritage, we should try to remove the print wherever they are...
08:36:10 <SerenaFeng> +1
08:36:20 <ollivier> morgan_orange: true we should fix that. I will do
08:36:45 <jose_lausuch> ok
08:36:54 <ollivier> jose_lausuch: at least one case you can't remove but I added the flush call
08:37:14 <morgan_orange> any other topic for Colorado?
08:37:27 <morgan_orange> bug/issue/concern/fear/
08:37:27 <jose_lausuch> when is the release date?
08:37:33 <morgan_orange> 22th of September
08:37:37 <jose_lausuch> ok
08:37:43 <jose_lausuch> still a lot of read in the reporting page :)
08:37:50 <morgan_orange> #info config guide and dev guide reviewed by opnfvdocs
08:38:29 <morgan_orange> scoring are not bad, feature projects triggering fail are identified
08:38:47 <morgan_orange> some scenarios already green
08:38:55 <morgan_orange> BTW yardstick adopted also our reporting
08:39:02 <morgan_orange> http://testresults.opnfv.org/reporting/yardstick/release/master/index-status-joid.html
08:39:49 <jose_lausuch> cool
08:40:10 <morgan_orange> #info dashboard will not be done by bitergia, so we will have to adapt our own (not priority / scenarios) but will be needed
08:40:24 <morgan_orange> #topic vote for 2 interns projects
08:40:26 <SerenaFeng> okey, I will do that
08:40:49 <morgan_orange> old topic but I would like to create 2 asap (request from academic)
08:40:59 <morgan_orange> I suggested several ones in a previous mail
08:41:28 <jose_lausuch> what topics ?
08:41:29 <morgan_orange> any proposal from your side (3 months intern)
08:41:52 <jose_lausuch> we have almost everything covered by all the contributors
08:42:00 <morgan_orange> #link https://wiki.opnfv.org/display/DEV/Intern-projects-page
08:42:05 <jose_lausuch> I would say as a possible topic/task, integration of VNFs
08:42:06 <morgan_orange> it is for the future...and just interns
08:42:25 <morgan_orange> my suggestions were around new VNFs
08:42:30 <jose_lausuch> for example JOID will have a lot of things for vnfs with charms
08:42:38 <jose_lausuch> there is an email around
08:42:39 <morgan_orange> or the vPing mentioned by Tim in a Jira for D
08:42:45 <jose_lausuch> someone wants to integrate a vnf
08:42:54 <morgan_orange> yes the vEPC from OAI
08:43:02 <jose_lausuch> exactly
08:43:08 <jose_lausuch> that could be 1 example of task for intern
08:43:28 <morgan_orange> if no objection, I will suggest 2 topics around VNF integration
08:43:58 <morgan_orange> Ok if you have idea you want to share offline, send a mail
08:44:08 <jose_lausuch> no objections
08:44:12 <morgan_orange> #action morgan_orange create 2 topics related to new VNF integration
08:44:35 <morgan_orange> #topic preparation of D release
08:44:50 <morgan_orange> already started a little bit (see patch Serena)
08:44:58 <morgan_orange> how shall we organize ourselves?
08:45:03 <morgan_orange> I would suggets to use the wiki page
08:45:15 <morgan_orange> https://wiki.opnfv.org/display/functest/Functextnexttaks
08:45:52 <morgan_orange> I see a big task related to templating/case abstration
08:46:05 <morgan_orange> before patching, it could make sense to share a high level view in the wiki
08:46:20 <jose_lausuch> ok
08:46:38 <morgan_orange> I would like also to contact SNAPS to start considering integrating their framework...as it was already object oriented
08:46:56 <jose_lausuch> ollivier: could you explain your idea of case abstraction to get rid of exec_tests.sh?
08:47:01 <jose_lausuch> I mean, in the wiki
08:47:01 <morgan_orange> there are also sveral threads on new VNFs
08:47:06 <jose_lausuch> it would be good to capture it
08:48:02 <ollivier> jose_lausuch: Yes (after finishing the pending tasks :))
08:48:29 <morgan_orange> yardstick has also some plan about an API to trigger the test, I think we should invite them to try to have a consistant approach
08:48:37 <morgan_orange> maybe a meeting during OpenStack summit in barcelona
08:48:57 <jose_lausuch> sure, this is not top prio
08:49:15 <morgan_orange> so do not hesitate to initiate the reflexions in D release in the wiki page
08:49:31 <morgan_orange> we may have a lso a meeting in Barcelona
08:49:44 <morgan_orange> and if needed organize a meetup in britanny (as we did in Espoo)
08:50:03 <morgan_orange> the idea is to have the roadmap ~ clear for November?
08:50:55 <jose_lausuch> yep
08:50:56 <jose_lausuch> ok
08:51:41 <morgan_orange> JuhaHaapa: Juhak viktor_t raghu_ May-meimei SerenaFeng jose_lausuch ollivier boucherv_orange ...1 or 2 words on your view of Functest for D release?
08:52:12 <SerenaFeng> I will make some suggestions on the wiki page
08:52:16 <morgan_orange> lhinds maybe we can share the link for security
08:52:28 <morgan_orange> as you already intiated something
08:53:02 <morgan_orange> ok let's wikize but do not be shy...
08:53:06 <lhinds> for d-release morgan_orange ?
08:53:10 <raghu_> ok
08:53:11 <jose_lausuch> I would say also code improved
08:53:18 <morgan_orange> lhinds yes
08:53:32 <lhinds> sure, makes sense
08:53:38 <morgan_orange> jose_lausuch: yes unit test especially for the util functions...but now that we have real developpers with us...
08:53:59 <jose_lausuch> unit test could be another task for an intern :)
08:54:09 <morgan_orange> it was also a suggestion...
08:54:46 <morgan_orange> moreove Mark is managing intern programs ...we may invite him / unit test management in Storperf
08:54:51 <morgan_orange> #topic AoB
08:55:13 <morgan_orange> #info committer vote to be closed this week
08:56:12 <morgan_orange> regarding the plugfest in UNH, not sure we have a Functest representative now...not sure it is as mandatory as it was for plugfest #1 but it makes sense to be there...
08:57:04 <jose_lausuch> when is it?
08:57:12 <morgan_orange> beginning of December
08:57:14 <jose_lausuch> http://events.linuxfoundation.org/events/opnfv-plugfest
08:57:49 <morgan_orange> looks like Plugfest in Dead Poets Society university... :)
08:58:11 <jose_lausuch> :)
08:58:41 <morgan_orange> any other topic you want to share...
08:58:53 <jose_lausuch> yep, as info
08:59:02 <jose_lausuch> https://hub.docker.com/r/opnfv/functest/
08:59:11 <jose_lausuch> I added a tag for our docker
08:59:19 <jose_lausuch> in the Full Description field
08:59:22 <jose_lausuch> if you click there
08:59:38 <jose_lausuch> you'll see all the info of our container with the different layers
08:59:39 <jose_lausuch> https://microbadger.com/#/images/opnfv/functest
09:00:03 <jose_lausuch> that gives an idea about how the image is structured and how much size it takes each layer in the image
09:00:40 <morgan_orange> good point ...shall we change also the reference in Colorado docker file to point only to colorado report?
09:00:55 <morgan_orange> shall we add a -b colorado for all the internal repos?
09:01:03 <jose_lausuch> what do you mean?
09:01:04 <jose_lausuch> where?
09:01:12 <jose_lausuch> ah when we clone the repos
09:01:12 <ollivier> yes I think so
09:01:13 <jose_lausuch> mmm
09:01:13 <morgan_orange> https://git.opnfv.org/cgit/functest/tree/docker/Dockerfile
09:01:19 <jose_lausuch> good point
09:01:29 <jose_lausuch> but we can do that only in our colorado branch.. not in master
09:01:30 <jose_lausuch> right?
09:01:35 <morgan_orange> yes
09:01:40 <ollivier> We should manage 2  Dockerfiles
09:01:44 <morgan_orange> we fixed the branch for rally and temepst
09:02:03 <morgan_orange> ollivier: good point
09:02:24 <morgan_orange> at the moment on colorado we use stable (in master latest..)
09:02:45 <jose_lausuch> ok
09:02:48 <ollivier> morgan_orange: maybe 3. another for Debian testing to decreasing OS size :)
09:02:48 <jose_lausuch> we can do that
09:03:25 <morgan_orange> #action jose_lausuch morgan_orange check Colorado docker consistancy and fix branch for OPNFV internal repos in Colorado(stable) docker
09:04:00 <morgan_orange> ok any other topic?
09:04:51 <morgan_orange> if no it is OK for today
09:05:00 <morgan_orange> I merged the logger related patches
09:05:03 <morgan_orange> have a good week
09:05:12 <morgan_orange> and enjoy D release...do not be shy...
09:05:17 <morgan_orange> #endmeeting