08:00:24 <morgan_orange> #startmeeting Functest weekly meeting June 7th
08:00:24 <collabot`> Meeting started Tue Jun  7 08:00:24 2016 UTC.  The chair is morgan_orange. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:00:24 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:00:24 <collabot`> The meeting name has been set to 'functest_weekly_meeting_june_7th'
08:00:30 <morgan_orange> #topic call role
08:00:34 <morgan_orange> #info morgan Richomme
08:01:04 <SerenaFeng> #info SerenaFeng
08:01:15 <jose_lausuch> #info Jose Lausuch
08:01:30 <morgan_orange> #info agenda: https://wiki.opnfv.org/display/functest/Functest+Meeting
08:01:52 <morgan_orange> juha and Cedric will not join today but I got some update to share
08:02:13 <morgan_orange> #topic action point follow up
08:02:18 <lhinds> #info lhinds
08:02:21 <morgan_orange> #link http://ircbot.wl.linuxfoundation.org/meetings/opnfv-testperf/2016/opnfv-testperf.2016-05-31-08.01.html
08:02:43 <morgan_orange> #info AP1: we did not dive deeply into Apex logs, we were a bit optimistic last week
08:02:52 <morgan_orange> #info Apex tempest success rate = 63%
08:03:00 <morgan_orange> #link http://testresults.opnfv.org/reporting/functest/release/index-tempest-apex.html
08:03:14 <morgan_orange> #info troubleshooting to be planned on Tempest and ODL
08:03:54 <morgan_orange> #info AP2: Cedric had a llok at ODL tests: his conclusion refactoring is needed as some tests are not relevant and some will be needed
08:04:07 <morgan_orange> #info he planned to refactor incrementally (no risk / regarding the freeze)
08:04:27 <morgan_orange> that is basically somehow what we had in mind for Brahmaputra
08:04:46 <morgan_orange> #info AP3: Juha answered during last week meeting, so AP close
08:05:22 <morgan_orange> #info AP4: May-meimei are you there? any update on gsutils on all CI PODs?
08:06:23 <morgan_orange> #info AP5: no update on domain coverage page https://wiki.opnfv.org/display/functest/Testing+Coverage but discussion connected to test slicing
08:06:40 <morgan_orange> #info proposal shared with testing community https://wiki.opnfv.org/display/functest/Testing
08:06:52 <morgan_orange> #info slicing under review, discussion planned on thursday
08:07:21 <morgan_orange> #info AP6: did anyone contact his/her ETSI contact?
08:07:43 <jose_lausuch> nope
08:07:51 <morgan_orange> nope also from my side
08:08:01 <morgan_orange> ok let's keep it open :)
08:08:03 <SerenaFeng> nope
08:08:27 <morgan_orange> #info AP7: done SerenaFeng has access to the machine, we will discuss next steps in next topic
08:08:37 <morgan_orange> end for action items
08:08:48 <morgan_orange> #topic Sprint #8 follow-up
08:09:00 <morgan_orange> #link https://jira.opnfv.org/secure/RapidBoard.jspa?rapidView=59
08:09:21 <morgan_orange> 2 days remaining...I will create a new sprint until feature freeze once Sprint #8 done
08:10:01 <morgan_orange> there are still lots of open Jiras (the one on integration have been included in Sprint 8) but they could have been put in later Sprint
08:10:07 <jose_lausuch> I will move a task to d-river
08:10:09 <morgan_orange> any update on our internal test case
08:10:43 <jose_lausuch> internal test case?
08:11:07 <morgan_orange> lhinds: security_scan
08:11:24 <morgan_orange> http://testresults.opnfv.org/reporting/functest/release/master/index-status-apex.html
08:11:39 <morgan_orange> security scan is now part of the reporting even if no test evaluate so far
08:11:40 <lhinds> hit some delays in testing, as the machine I was on got taken out to have a memory change (it failed)..
08:12:05 <lhinds> I now build an internal one which is running apex, so I will get testing today
08:12:16 <morgan_orange> cool
08:12:40 <lhinds> so might be the odd code tweak coming in this week if I find anything, but things should get moving again
08:13:19 <morgan_orange> I think healthcheck and secruity scan will be the 2 new internal testcases for C (even if we refactor Tempest/Rally and created new test cases) + new feature project suite
08:13:29 <lhinds> I need to have it working for the summit too
08:13:58 <morgan_orange> we will discuss on the summit in next topic, but we may plan integration session there
08:14:30 <morgan_orange> jose_lausuch: for vIMS offline, I think we will also push it to D, it is not the top priority...
08:14:41 <jose_lausuch> ok, no prob
08:14:55 <jose_lausuch> is valentin there?
08:15:10 <morgan_orange> #info security scan:  ome delays in testing, as the machine I was on got taken out to have a memory change (it failed), testing today
08:15:17 <morgan_orange> jose_lausuch: yes he is back..
08:15:22 <jose_lausuch> Valentin_Orange: hello
08:15:33 <jose_lausuch> I've seen that it downloads an image from internet
08:15:33 <morgan_orange> and he will be in Berlin
08:15:38 <jose_lausuch> if that's the only thing
08:15:44 <jose_lausuch> we can think of putting that image in the container
08:16:35 <morgan_orange> i suggets to put in Sprint 9 only our internal cases and keep all the intergation related task for the one after
08:16:55 <morgan_orange> #info Functest versus Feature project : http://picpaste.com/BrahmaFunctestVsfeatureProject-AqTnwvgv.png
08:17:25 <morgan_orange> #info all the information not known yet, Summit will be the occasion to finalize that. A 1 h session is planned during the design summit on feature integration
08:18:02 <morgan_orange> #info table shared with David McBride
08:18:41 <morgan_orange> Any question/concern/problem regarding our Jiras?
08:19:18 <morgan_orange> #topic Berlin Summit
08:19:19 <jose_lausuch> no
08:19:32 <morgan_orange> #info lots of slots prebooked for Functest during the Design Summit
08:19:46 <jose_lausuch> morgan_orange: raimond sent me an email
08:19:47 <morgan_orange> #link https://wiki.opnfv.org/display/EVNT/Berlin+Design+Summit+Planning
08:19:52 <jose_lausuch> about the session I proposed
08:19:56 <jose_lausuch> that we could maybe merge it
08:20:01 <morgan_orange> #link https://wiki.opnfv.org/display/EVNT/Berlin+Design+Summit+Project+Breakouts+and+Hacking
08:20:29 <morgan_orange> it was the hands on session, rigth?
08:20:33 <jose_lausuch> yes
08:20:50 <jose_lausuch> so, no hands on :)
08:20:50 <morgan_orange> and where does he want to merge it?
08:20:56 <morgan_orange> for the moment we have
08:21:04 <jose_lausuch> he didnt put it in the schedule
08:21:50 <jose_lausuch> he said that there are already different functest related sessions
08:21:59 <jose_lausuch> they decided not to add another one
08:21:59 <morgan_orange> Monday: 10h30-11: Plugfest, 11h-11h30: Functest, 11h30-12h: Conversation with the testing community, 16h-17h feature project integration, 17h-17h30 Functest Colorado
08:22:11 <jose_lausuch> so no hands on :)
08:22:14 <jose_lausuch> which is fine
08:23:21 <morgan_orange> Tuesday 11h-11h30 Test API, 11h30-12h0 CI/CD panel, 13h-13h30 VNF discussion*, 13h30-14h0 test API* , 14h-14h30 Dashboard/ELK*, 15h15-15h45 D release*
08:23:31 <morgan_orange> * = breakout session
08:23:49 <morgan_orange> are we OK with the breakout sessions, do we want to add/modify some
08:23:58 <jose_lausuch> we can't add more stuff
08:24:05 <morgan_orange> agree....
08:24:22 <morgan_orange> #info Monday: 10h30-11: Plugfest, 11h-11h30: Functest, 11h30-12h: Conversation with the testing community, 16h-17h feature project integration*, 17h-17h30 Functest Colorado*
08:24:29 <morgan_orange> #info Tuesday 11h-11h30 Test API, 11h30-12h0 CI/CD panel, 13h-13h30 VNF discussion*, 13h30-14h0 test API* , 14h-14h30 Dashboard/ELK*, 15h15-15h45 D release*
08:24:36 <morgan_orange> #info *= break out sessions
08:25:03 <SerenaFeng> 11h-11h30 Test API
08:25:03 <morgan_orange> for the Functest presentation, I merge the pres, feel free to complete it
08:25:06 <SerenaFeng> 13h30-14h0 test API*
08:25:12 <jose_lausuch> which presentation?
08:25:19 <SerenaFeng> there are two test API?
08:25:30 <morgan_orange> one is a presentation, one is a breakout session
08:25:40 <morgan_orange> presentation is a little bit more formal, with slides
08:25:57 <morgan_orange> breakout is 30 minutes to perform demo, answer to other project, help them to use it
08:26:10 <SerenaFeng> okey
08:26:46 <morgan_orange> for the functets presentaiton (Monday 11h30-12h) => https://git.opnfv.org/cgit/functest/tree/docs/com/pres/summit-Berlin.html
08:27:02 <morgan_orange> for test API => https://gerrit.opnfv.org/gerrit/#/c/15143/
08:27:17 <morgan_orange> for test APi, SerenaFeng did everything...I just bootstraped the sldies :)
08:27:33 <jose_lausuch> wait
08:27:35 <jose_lausuch> monday 11:30?
08:27:36 <morgan_orange> for conversation with the testing community => https://gerrit.opnfv.org/gerrit/#/c/15087/
08:27:39 <SerenaFeng> for now I don't have time to have a look
08:27:54 <SerenaFeng> I will review it after the update work is done
08:27:56 <morgan_orange> SerenaFeng: no problem...we can see your Jira and code activity...
08:28:01 <jose_lausuch> OPNFV Functest From Brahmaputra to C release: many rivers to cross… ?
08:28:11 <morgan_orange> jose_lausuch: yes
08:28:15 <jose_lausuch> ok
08:28:30 <jose_lausuch> 11:00 - 11:30
08:28:40 <jose_lausuch> not 11:30-12
08:29:01 <morgan_orange> #action jose_lausuch viktor Juha complete https://git.opnfv.org/cgit/functest/tree/docs/com/pres/summit-Berlin.html
08:29:15 <jose_lausuch> morgan_orange: how can we see that in the presentation format?
08:29:18 <morgan_orange> yep sorry 11h30-12h00 is conversation with the testing community => presentation of all the testing project
08:30:10 <jose_lausuch> I think we should use the opnfv template, at least that's what I've been told
08:30:10 <morgan_orange> jose_lausuch: you mean is it the official format?
08:30:35 <jose_lausuch> morgan_orange: yes, the reveal format
08:30:37 <jose_lausuch> not code :)
08:30:42 <morgan_orange> I did not see any template
08:31:17 <morgan_orange> as far as I remember, there is a doc meeting today, Sofia will report my request
08:32:00 <jose_lausuch> ok
08:32:09 <morgan_orange> and so your hands'on sessioncould be merge with the 1h slot of feature test integration?
08:32:24 <jose_lausuch> mmmm
08:32:27 <jose_lausuch> which slot?
08:32:48 <jose_lausuch> the breakout session?
08:32:59 <morgan_orange> Monday 16-17h feature project integration (2 tables)
08:33:01 <jose_lausuch> modanyt16:00-16:30
08:33:10 <jose_lausuch> ok
08:33:12 <morgan_orange> I put 2 slots in a row
08:33:16 <morgan_orange> raw
08:33:31 <jose_lausuch> what is the intention of these 2 slots?
08:33:31 <morgan_orange> we will see on site...
08:33:46 <jose_lausuch> maybe we need it for other things
08:33:56 <jose_lausuch> we need that slot to talk to the fesature projects people :)
08:34:01 <morgan_orange> the intention was to attract the feature project (the one mentioned in the table shared above) and see with them how to integrate their test in our system
08:34:02 <jose_lausuch> its ok
08:34:20 <morgan_orange> but somehow it corresponds to demo
08:34:27 <morgan_orange> to see what they have to do on their side..
08:34:54 <morgan_orange> jose_lausuch: we have also a presentation for the summit, I did not start mines..but it is planned for this week...
08:34:54 <jose_lausuch> I see it more like a discussion and if they want more details, we can explain them the code/repo
08:35:05 <jose_lausuch> yes ,I know
08:35:06 <morgan_orange> jose_lausuch: ok
08:35:41 <morgan_orange> ok any more question on Berlin, which looks already exhausting 2 weeks before...
08:36:18 <morgan_orange> #topic Status on test API refactoring
08:36:18 <jose_lausuch> more questions will come :)
08:36:38 <morgan_orange> #info SerenaFeng did a great job (deserve the quartely awards...)
08:37:09 <morgan_orange> #info tets API fully refactored, including unit tests, backup/restore/deployment scripts, swagger framework for automatic documentation
08:37:24 <morgan_orange> #info next step migration on testresults.opnfv.org
08:37:33 <morgan_orange> SerenaFeng: I planned to do this after the meeting
08:37:33 <SerenaFeng> yeah,
08:37:40 <morgan_orange> I planned to keep the old tables first
08:37:46 <SerenaFeng> the kibana access is a issue
08:37:53 <morgan_orange> duplicate them/renamed them/adapt them
08:38:10 <SerenaFeng> the rename script is ok now
08:38:11 <morgan_orange> then stop existing process, update releng and restart yours
08:38:22 <SerenaFeng> I haven't put them to the repo
08:38:41 <morgan_orange> I can do it directly in the DB, I did it locally on my machine yesterday
08:38:41 <SerenaFeng> whick include rename the collection test_xxx to xxx
08:38:56 <SerenaFeng> and change creation_date in results to start_date
08:39:16 <morgan_orange> ok so as you prefer, manual operation or script
08:39:30 <morgan_orange> I think we also should add some changes in test_criteria
08:39:34 <morgan_orange> failed => FAILED
08:39:38 <morgan_orange> passed => PASS
08:39:43 <morgan_orange> to be consistent with test criteria
08:40:08 <SerenaFeng> in this way, I prefer script
08:40:28 <SerenaFeng> I know the way to change the content of a field
08:40:29 <morgan_orange> OK, so I wait for your script if you prefer
08:40:47 <SerenaFeng> if we do i manually, we will do it always manually
08:40:50 <SerenaFeng> ok
08:41:00 <SerenaFeng> you can tell all the changes
08:41:09 <SerenaFeng> then I can add them to the script
08:41:11 <morgan_orange> yesterday it took me 10 minutes but you are right for the future and the history, scripting is always better
08:41:43 <morgan_orange> I will also make some change in the push2DB function in the different testcases
08:41:56 <SerenaFeng> for now we can do to with access to the testresults.opnfv.org
08:42:07 <SerenaFeng> next version we can use ansible to do the update
08:42:28 <morgan_orange> ok would be great
08:42:34 <SerenaFeng> in that way, we don't need to login any more
08:43:03 <SerenaFeng> and I am considerring docker-ize testAPI, the update will be more convenient
08:43:14 <morgan_orange> do you think we can introduce the change to the other test projects during the thursday meeting, or do you prefer to wait for berlin?
08:43:49 <SerenaFeng> both ok
08:44:36 <morgan_orange> ok, let me know if you can/want to connect on thursday (quiete late for you), if not I can do it and indicate that you will be in berlin
08:45:05 <SerenaFeng> ok
08:45:16 <morgan_orange> #action SerenaFeng finalize migration scripts (no manual operation) before switching APi on testresults
08:45:38 <morgan_orange> #info switch should be possible very soon
08:45:41 <SerenaFeng> and who will do the kibana adaptions ?
08:45:56 <morgan_orange> #info SerenaFengindicates possible evolution using container and ansible to simplify depliyment/update/..
08:46:29 <SerenaFeng> and fix the kibana dashboard cannot loggin problem?
08:46:31 <morgan_orange> for Kibana, Juraj makes a presentation during the summit, but I think we will have to do the adaptations...
08:46:53 <morgan_orange> for the logging, we may ask the community to see who did anything, if nobody it is maybe a soft issue
08:47:04 <SerenaFeng> after the migration we need the way to test it is work fine, the dashboard is a good way
08:47:15 <morgan_orange> #action morgan_orange send mail to the community to ask who did any change on kibana
08:47:43 <morgan_orange> #action SerenaFeng morgan_orange consider ELK adapations once switch done
08:48:01 <morgan_orange> ok for this part
08:48:04 <morgan_orange> any question?
08:48:10 <jose_lausuch> nope
08:48:20 <jose_lausuch> SerenaFeng is doing a great job
08:48:30 <morgan_orange> #topic AoB
08:48:37 <morgan_orange> any other topic you would like to share here
08:48:42 <jose_lausuch> yes
08:48:45 <jose_lausuch> https://wiki.opnfv.org/display/multisite/Kingbird+integration
08:48:49 <jose_lausuch> look for "functest"
08:48:58 <jose_lausuch> I think we will have to support this activity soon
08:49:22 <morgan_orange> as far as I read minutes of their minutes, they use Functest as baseline and made some adapations for their project
08:49:36 <jose_lausuch> they install some libraries and use tempest
08:49:38 <morgan_orange> so that is a good way to do it but yes we will have to integrate that into repo
08:49:42 <jose_lausuch> have they contacted you?
08:49:51 <morgan_orange> no regarding the table I shared
08:50:03 <jose_lausuch> if they wat integration in CI we should align...
08:50:09 <jose_lausuch> otherwise everything will be manual
08:50:26 <morgan_orange> they have "no" everywhere...project not declared in the DB, no test case description and no JIRA created..
08:50:46 <morgan_orange> #action morgan_orange contact multi site to ask for test, test case declaration + Jira for integration
08:51:07 <morgan_orange> #action morgan_orange contact PTL of feature projects for which one of this 3 elements is missing
08:51:21 <jose_lausuch> according to this http://picpaste.com/pics/BrahmaFunctestVsfeatureProject-AqTnwvgv.1465289473.png
08:51:41 <jose_lausuch> althoug its not our task to go behind anyone
08:51:50 <jose_lausuch> let's try to doublecheck before saying "no"
08:52:06 <morgan_orange> I have another concern, regarding CI Pipeline.... fuel, joid no run since 10 days..
08:52:15 <jose_lausuch> yes
08:52:22 <morgan_orange> we moved to master but we do not have lots of "stable" master
08:52:24 <jose_lausuch> everything is red in CI...
08:52:30 <jose_lausuch> deployments, tests...
08:52:38 <jose_lausuch> there is nothing stable I would say
08:52:47 <jose_lausuch> it's installer integration time..
08:52:50 <morgan_orange> we have still some blue balls...
08:52:53 <morgan_orange> compass
08:53:10 <morgan_orange> and apex daily is running (troubleshooting time there)
08:53:29 <morgan_orange> we have an issue with odl_l3 scenario => healthcheck fails
08:53:50 <morgan_orange> https://build.opnfv.org/ci/job/functest-apex-apex-daily-master-daily-master/9/console
08:53:56 <jose_lausuch> ok, someone to troubleshoot that
08:54:04 <jose_lausuch> I have other AoB as well
08:54:25 <morgan_orange> but you are right it is installer time...
08:54:27 <morgan_orange> yep
08:54:28 <jose_lausuch> gbpvpn test cases defined in yardstick will be integrated in functest (I'll implement them)
08:54:45 <jose_lausuch> and probably we will move SFC tests as well, but it is not 100%
08:54:53 <jose_lausuch> when is the deadline for test implementation ready?
08:54:56 <jose_lausuch> 7/7/16?
08:55:17 <morgan_orange> #info bgpvpn tests to be moved from yardstick to Functest ...pobably SFC (depending on the complexity) to be more consisten with the aggreed view (Functional testing = Functets)
08:55:59 <morgan_orange> officially the feature freeze is beginning of July but here we can consider that it is integration freeze so it is later
08:56:28 <morgan_orange> #info some concern on the CI stability at the moment: no run on fuel/joid on Master since more than 10 days
08:56:40 <jose_lausuch> I got this question many times
08:56:53 <jose_lausuch> what is the difference between test integration ready and test implementation ready?
08:57:04 <jose_lausuch> almost 100% of the cases integration=implementation
08:57:14 <morgan_orange> yes and no
08:57:26 <morgan_orange> for ex ovno they developped their test suite for brahmaputra
08:57:30 <morgan_orange> we never integrated them
08:57:45 <morgan_orange> so for me test implementation is up to the feature project to create their scripts
08:58:04 <jose_lausuch> but if they have to create them in yardstick/functest...
08:58:08 <morgan_orange> integration ready is the work between feature and test project for the integration into CI pipeline/dashboarding
08:58:09 <jose_lausuch> its integration at the same time :)
08:58:15 <morgan_orange> yes
08:58:17 <jose_lausuch> but anyway
08:58:19 <jose_lausuch> its about naming
08:58:48 <morgan_orange> usuaully whe it is internal test case we do both at the same time
08:59:02 <morgan_orange> but typically multi site is working on test implementation
08:59:06 <morgan_orange> and did not start test integration
08:59:42 <morgan_orange> we distinguished that because feature projects are coding until the last day befroe the freeze
08:59:49 <jose_lausuch> ok
08:59:58 <morgan_orange> so integration should have been immediate, which is usually not possible...
09:00:17 <morgan_orange> any other info to share this week?
09:00:28 <morgan_orange> if no we can close the conf
09:00:51 <jose_lausuch> nope
09:00:52 <jose_lausuch> ok
09:00:59 <SerenaFeng> ok
09:01:21 <morgan_orange> SerenaFeng: let me know when the script is ready, I can perform backup/change/ as sson as you give me your green light
09:01:31 <SerenaFeng> ok
09:01:37 <morgan_orange> thanks everybody, have a good week
09:01:40 <morgan_orange> #endmeeting