08:01:20 <jose_lausuch> #startmeeting Functest weekly meeting July 11th 2017
08:01:20 <collabot`> Meeting started Tue Jul 11 08:01:20 2017 UTC.  The chair is jose_lausuch. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:01:20 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:01:20 <collabot`> The meeting name has been set to 'functest_weekly_meeting_july_11th_2017'
08:01:20 <mbuil> morgan_orange: yes, the scenario name changed! :O
08:01:24 <jose_lausuch> #topic role call
08:01:34 <jose_lausuch> #info Jose Lausuch
08:01:34 <morgan_orange> #info Morgan
08:01:38 <jose_lausuch> #chair morgan_orange
08:01:38 <collabot`> Current chairs: jose_lausuch morgan_orange
08:02:13 <jose_lausuch> #info agenda for today: https://wiki.opnfv.org/display/functest/Functest+5.+Meeting#Functest5.Meeting-11/07(8UTC)
08:02:26 <boucherv> #info Valentin Boucher
08:02:44 <morgan_orange> shall we add a topic for Danube 3.0 (it is today) or Euphrates MS 3.2 (test with a SDN controler :))
08:03:22 <jose_lausuch> yes
08:03:45 <jose_lausuch> but I'm sure folks are more interested to talk about alpine :)
08:04:33 <OPNFV-Gerrit-Bot> Merged functest: Allow regex for blacklist scenarios/installers  https://gerrit.opnfv.org/gerrit/37073
08:04:34 <morgan_orange> another possible topic would be the testapi ...new restart needed this morning..
08:04:51 <morgan_orange> but let's start with Alpine
08:05:00 <jose_lausuch> ollivier: ping?
08:05:01 <jose_lausuch> #topic Action Point follow-up
08:05:19 <jose_lausuch> not so many people in the meeting today
08:05:29 <Martin_nokia> Sorry, I was bothering ollivier with my questions :)
08:05:33 <ollivier> pong
08:05:38 <LindaWang> #info Linda Wang
08:05:44 <ollivier> #info Cédric
08:05:47 <jose_lausuch> #info AP ollivier create a wiki and list all the remaining issues that prevent us from fully using Alpine
08:05:50 <ollivier> Hello, World!
08:05:51 <jose_lausuch> #info wiki created with instructions how to run the new containers https://wiki.opnfv.org/display/functest/Running+new+Functest+containers
08:05:57 <jose_lausuch> hello ollivier
08:06:04 <jose_lausuch> #info AP ollivier publish Alpine Dockerfile in /docker/alpine/Dockerfile
08:06:08 <jose_lausuch> #info DONE. Testing and feedback in progress
08:06:14 <jose_lausuch> #info AP ollivier check tempest version to be used to run against Ocata.
08:06:21 <jose_lausuch> #info DONE. According to upper constraints, it should be 14.0.0
08:06:27 <jose_lausuch> anything to add ollivier ?
08:06:31 <ollivier> Yes
08:06:35 <jose_lausuch> we have a topic
08:06:45 <ollivier> I think the last errors are not related to Alpine
08:06:52 <jose_lausuch> #topic Alpine testing status
08:07:16 <ollivier> First I would appreciate we merge the pathes as it ease building the containers
08:07:30 <ollivier> (we download functest from amster)
08:07:40 <ollivier> Two testcases are failing snaps and refstack
08:07:46 <jose_lausuch> https://gerrit.opnfv.org/gerrit/#/c/36963/12 ?
08:08:11 <morgan_orange> #info  last errors (snaps / refstack) are not related to Alpine
08:08:16 <ollivier> Yes https://gerrit.opnfv.org/gerrit/#/q/topic:alpine+(status:open)
08:08:33 <morgan_orange> #info investigation in progress on snaps and refstack side
08:08:37 <ollivier> I will allow to automatically build the containers
08:08:42 <jose_lausuch> I think we can merge now as they are accepted (>=2 +2s)
08:08:47 <OPNFV-Gerrit-Bot> Merged functest: Add lesser containers based on Alpine  https://gerrit.opnfv.org/gerrit/36963
08:08:58 <ollivier> regarding snaps: https://gerrit.opnfv.org/gerrit/#/c/37137/
08:09:11 <OPNFV-Gerrit-Bot> Merged functest: Add upper-constraints.txt for Functest  https://gerrit.opnfv.org/gerrit/37077
08:09:12 <ollivier> Ubuntu container should fail as well
08:09:41 <LindaWang> ollivier: why?
08:09:48 <jose_lausuch> #info fixed bug in ansible test case https://gerrit.opnfv.org/gerrit/#/c/37137/
08:10:02 <ollivier> It depends if default security group allows ssh or not
08:10:07 <OPNFV-Gerrit-Bot> Merged functest: Download Functest's upper-constraints.txt  https://gerrit.opnfv.org/gerrit/37097
08:10:15 <ollivier> if not, I bet it fails
08:10:17 <OPNFV-Gerrit-Bot> Merged functest: Define CMD in smoke and healthcheck containers  https://gerrit.opnfv.org/gerrit/37105
08:10:23 <morgan_orange> for snaps, would it make sense to ask Steven to grant the +2 to ollivier, jose_lausuch and LindaWang ..I think he is the only one to have the right to merge..so if he is in vacations...functest can be blocked
08:10:49 <jose_lausuch> morgan_orange: yes, someone should have +2 rights just in case
08:10:55 <morgan_orange> strong upper-constraint ...
08:11:00 <ollivier> :)
08:11:02 <jose_lausuch> I had to send him once an SMS to merge something while he was on vacation
08:11:11 <jose_lausuch> and I felt so bad about it
08:11:26 <ollivier> Please give me an action point to automatically build the alpine container now
08:11:43 <jose_lausuch> #action ollivier docker alpine image automatic build
08:11:59 <morgan_orange> #action jose_lausuch ask Steven to promote news snaps committer
08:12:03 <ollivier> It will be great if I can sync the official OPNFV github repo
08:12:21 <jose_lausuch> I think Linda has done a lot of work on snaps repo, maybe she can be the candidate :)
08:12:24 <ollivier> jose_lausuch: Could you please send the request as proposed by email yesterday?
08:12:32 <ollivier> Else I must sync by hand my fork repo
08:12:42 <jose_lausuch> #action sync with opnfv helpdesk for automatic builds from opnfv github mirror
08:12:42 <LindaWang> ollivier: wow.
08:12:46 <jose_lausuch> #undo
08:12:46 <collabot`> Removing item from minutes: <MeetBot.ircmeeting.items.Action object at 0x26a4e90>
08:12:52 <jose_lausuch> #action jose_lausuch sync with opnfv helpdesk for automatic builds from opnfv github mirror
08:13:07 <jose_lausuch> ollivier: yes, I will
08:13:17 <ollivier> thank you
08:13:33 <jose_lausuch> ollivier: do you know if the github hook in dockerhub can build multiple containers?
08:13:43 <ollivier> Yes. I'm ready
08:13:45 <jose_lausuch> I did that once but with only 1 Dockerfile
08:14:20 <ollivier> I tested. But I didn't want to publish the hack to build refs instead of master.
08:14:25 <jose_lausuch> ok
08:14:31 <ollivier> even in my github repo.
08:14:42 <jose_lausuch> btw
08:14:46 <ollivier> I will do this morning.
08:14:47 <jose_lausuch> how shall we publish the new images?
08:14:53 <jose_lausuch> since we already have
08:14:59 <jose_lausuch> opnfv/functest:latest or stable
08:15:06 <jose_lausuch> what will be the tag for the new ones?
08:15:22 <jose_lausuch> as we have 1 docker hub repo for functest
08:15:29 <ollivier> I would propose to keep ollivier/functest-core etc. as long as releng doesn't handle multiple containers
08:16:02 <jose_lausuch> so in the end we will have
08:16:12 <ollivier> I will automate the build. If your request is accepted (to sync official repo), it will work as well
08:16:15 <jose_lausuch> opnfv/functest-core:latest or stable or euphrates.1.0
08:16:24 <ollivier> yes
08:16:26 <jose_lausuch> opnfv/functest-smoke:latest or stable or euphrates.1.0
08:16:35 <jose_lausuch> then we need to request creating more repos for functest
08:16:48 <ollivier> Docker repo, yes
08:16:50 <jose_lausuch> how many do we need?
08:16:53 <jose_lausuch> yes, docker repos
08:17:06 <jose_lausuch> functest-core, functes-smoke, functest-features
08:17:07 <ollivier> features, components, vnfs
08:17:14 <jose_lausuch> let's list them here
08:17:18 <ollivier> one per suite + core for functest itself
08:17:37 <ollivier> then we could split more...
08:17:38 <jose_lausuch> #info new docker repos needed:   functest-core, functest-smoke,  functest-features, functest-components, functest-vnfs
08:17:46 <ollivier> healthcheck
08:18:05 <ollivier> it's listed on the gerrit change i think
08:18:19 <jose_lausuch> #info functest-healthcheck
08:18:19 <jose_lausuch> ok
08:18:20 <morgan_orange> note in components it is tempest and rally full ...the ondly difference with smoke will be the configuration of rally and tempest
08:18:22 <ollivier> https://gerrit.opnfv.org/gerrit/#/c/36963/
08:18:28 <jose_lausuch> #info 1 docker repo per tier
08:18:34 <jose_lausuch> well, that's right
08:18:39 <jose_lausuch> maybe we don't need functest-components
08:18:47 <boucherv> Are you sure for vnfs container ?
08:18:53 <ollivier> +1 for core. It's the key as all inherit from it
08:19:12 <LindaWang> jose_lausuch ollivier: i believe there is no need for functest-components
08:19:12 <ollivier> boucherv: Do you a special ims one :)
08:19:23 <boucherv> Because the VNF requierement are really differents between each VNF
08:19:41 <ollivier> I know buyt we have to improve it as well
08:19:50 <boucherv> ok
08:20:05 <jose_lausuch> for now, 1 container for all VNFs
08:20:07 <jose_lausuch> and then we will see
08:20:10 <jose_lausuch> right?
08:20:58 <morgan_orange> yes - we do not have so many VNFs... cloudify_ims, orchestra_ims, opera_ims, cloudify_vyos maybe juju_epc...
08:21:08 <ollivier> Yes. don't be afraid it will quite simple to split again. The main work is about defining requirements, upper requirements. Then it's just splitting the files and creating the right testcases.yaml
08:21:18 <jose_lausuch> ok
08:21:19 <morgan_orange> and for the moment on master only cloudify_ims
08:21:29 <jose_lausuch> ok
08:21:34 <jose_lausuch> anything else about alpine topic?
08:21:42 <jose_lausuch> good work ollivier
08:21:56 <morgan_orange> considering alpine testing next steps is to start testing features
08:21:57 <ollivier> I think it's right to time to switch to Alpine (testing)
08:21:59 <jose_lausuch> we will follow up next week to see what the progress is
08:22:13 <LindaWang> ollivier: i'd like to help refstack issues in Alpine
08:22:17 <jose_lausuch> #action jose_lausuch request a virtual deployment job for alpine testing
08:22:18 <morgan_orange> shall we allocate also the feature to split the load
08:22:20 <ollivier> CI/CD checks the ubuntu one.
08:22:55 <morgan_orange> do we have a good view of the features for Euphrates...
08:23:11 <jose_lausuch> morgan_orange: those that are enabled :)
08:23:12 <ollivier> The main point was to create all the requires files. Now I will init the next containers, clone ODL, get rid of tacker scripts...
08:23:30 <jose_lausuch> I bet that feature projects that really care about the test in CI, will realize if the test case is enabled or not
08:23:52 <ollivier> But I require your help to fix refstack and snaps
08:24:04 <ollivier> LindaWang: thank you Linda
08:24:24 <morgan_orange> yes but even. promise asked to be enabled but according to the repo stats, the activity is very low...
08:24:24 <LindaWang> Sure. I will try my best.
08:24:37 <ollivier> LindaWang: sure I will help you
08:25:10 <jose_lausuch> we can't block feature test if they want to be in CI because of the activity is slow
08:25:15 <jose_lausuch> but I get your point
08:25:21 <ollivier> Regarding the container, Promise adds a lot of requirements
08:25:35 <jose_lausuch> nodejs, right?
08:25:38 <morgan_orange> yes
08:26:16 <ollivier> I think we must oblige third party to follow new rules. I'm fixing the third party projects so much...
08:26:57 <ollivier> I think there is an action point about requirements too.
08:26:59 <jose_lausuch> but what can we do if they require new packages to be installed?
08:27:18 <jose_lausuch> yes, for example, domino test case does pip install in the domino script itself
08:27:21 <ollivier> First we must get rid of dedicated installation script
08:27:32 <ollivier> They must follow OpenSTack's requirements.
08:27:39 <jose_lausuch> #action jose_lausuch review all the enabled feature test if they perform any package installation
08:27:58 <ollivier> If the project hosts python scipt, they must provide a good python package as well
08:28:11 <jose_lausuch> yes
08:28:17 <ollivier> Quite trivial.
08:28:19 <jose_lausuch> some of them are simply shellscripts
08:28:25 <jose_lausuch> copper/domino,...
08:28:42 <ollivier> If they require nodejs. Why not but we rely on ubuntu or alpine package
08:28:59 <jose_lausuch> yes
08:29:10 <ollivier> Quite trivial rules...
08:29:29 <jose_lausuch> maybe we could put those rules in the wiki
08:29:39 <jose_lausuch> but don't want to action more :)
08:29:45 <jose_lausuch> can we move to next topic?
08:30:00 <ollivier> You can.
08:30:01 <ollivier> yes
08:30:06 <jose_lausuch> ok
08:30:11 <jose_lausuch> #topic Add images in Functest
08:30:17 <jose_lausuch> I'm not sure what you propose in Alpine
08:30:25 <jose_lausuch> but in Ubuntu image we have add_images and download_images
08:30:41 <jose_lausuch> add_images are injected in the docker build
08:30:50 <jose_lausuch> download images separated, to be added as a docker volume
08:30:52 <jose_lausuch> my suggestion was to get rid of add_images.sh
08:30:54 <ollivier> WXe must get rid of that
08:30:56 <jose_lausuch> what do you think?
08:31:04 <LindaWang> Actually download_images.sh is enough
08:31:16 <jose_lausuch> download images doesn't include cirros, right?
08:31:19 <ollivier> I think the containers should embed images and me must keep only one script to download
08:31:25 <ollivier> shouldn't
08:31:29 <ollivier> https://wiki.opnfv.org/display/functest/Running+new+Functest+containers
08:31:33 <jose_lausuch> yes, agree, shouldn't
08:31:33 <LindaWang> jose_lausuch:  no
08:31:45 <LindaWang> download_images has all images required
08:31:55 <ollivier> Yes but there is one issue in it
08:32:15 <ollivier> We should add argument about the tiers... For healthcheck we require only cirros
08:32:51 <ollivier> smoke requires Centos and ubuntu
08:33:03 <LindaWang> Oh, we have to add firewall_block_image.img into download_images, if we decide to remove add_images
08:33:05 <ollivier> (None requires Debian??)
08:33:20 <ollivier> True.
08:33:23 <jose_lausuch> yes
08:33:34 <jose_lausuch> so, do we agree on removing add_images.sh ?
08:34:10 <LindaWang> Only snaps_smoke needs Centos and ubuntu 14.04
08:34:11 <jose_lausuch> and tweak download_images to not download everything everytime? (e.g. add arguments, conditions, ..)
08:34:14 <ollivier> One or the other... Both have pros and cons. I think you should allow specifying a dest dir (default value the functest dir)
08:34:21 <LindaWang> bgpvpn need  ubuntu 16.04
08:34:38 <jose_lausuch> the idea is to have a docker volume with the images from a local download dir
08:34:39 <ollivier> add tiers as arg as well
08:34:59 <jose_lausuch> #info agreed to remove add_images.sh, container images shouldn't embed images
08:35:11 <ollivier> We must download md5sum as well
08:35:17 <jose_lausuch> #info add arguments in download_images.sh for different containers/images
08:35:33 <jose_lausuch> #info add md5sum to comply with ONFV security audit
08:35:46 <ollivier> to comply with standards :)
08:35:51 <jose_lausuch> as well
08:36:07 <jose_lausuch> ok
08:36:28 <jose_lausuch> I Will take the point of proposing removing add_images.sh script
08:36:38 <jose_lausuch> and then we will further think how to structure download_images.sh
08:36:52 <jose_lausuch> #action jose_lausuch propose a patch without add_images.sh
08:37:02 <jose_lausuch> anything else?
08:37:03 <ollivier> Could you check which one is called by gating
08:37:10 <jose_lausuch> gating?
08:37:39 <ollivier> We require download images in jjobs if ubuntu get rid of images
08:38:08 <jose_lausuch> I don't quite get your point
08:38:19 <jose_lausuch> yes, we require download_images.sh in CI currently
08:38:49 <ollivier> ok.
08:40:35 <morgan_orange> #topic Enabling cloudify_ims (and vnf) in daily
08:40:36 <jose_lausuch> we had a topic about vims
08:40:38 <jose_lausuch> #topic Enabling cloudify_ims (and vnf) in daily
08:40:39 <jose_lausuch> uh
08:40:41 <jose_lausuch> #undo
08:40:41 <collabot`> Removing item from minutes: <MeetBot.ircmeeting.items.Topic object at 0x266bad0>
08:40:49 <morgan_orange> #info config done
08:41:08 <morgan_orange> #info wait for master CI run going beyond smoke tests...
08:41:24 <jose_lausuch> #info vims enabled in daily jobs https://gerrit.opnfv.org/gerrit/#/c/36869/
08:42:00 <jose_lausuch> anything else you want to cmment?
08:42:02 <morgan_orange> #info contact with other people working on VNF onboarding tests (orchestra_ims juju_epc) => should be ready to start working on their integration on a community lab
08:42:27 <morgan_orange> LindaWang: do you know if soemone will look at opera_ims or due to ONAP changes, nothing is planned?
08:42:35 <morgan_orange> for Euphrates
08:42:45 <LindaWang> Do you mean from Huawei team?
08:42:57 <morgan_orange> yes I think Jelan worked on that for Dnaube
08:43:14 <morgan_orange> %s/Jelan/Helan/
08:43:19 <LindaWang> But Helen has left.
08:43:38 <LindaWang> Now I will check with compass team about that
08:43:45 <morgan_orange> ok
08:44:05 <jose_lausuch> #action LindaWang check with Compass team about opera_ims status due to ONAP changes
08:44:20 <morgan_orange> as there is apparently no more open-o scenario (waiting for an onap scenario?) I think this case will not be available for Euphrates, but if you can confirm with Comapss team, it will be great
08:44:36 <morgan_orange> #topic Logger (Valentin)
08:44:59 <jose_lausuch> #info there has been a discussion by email about having a user-friendly console output for testcases
08:45:32 <jose_lausuch> #info currently, we mute all the output on screen while a testcase is running, and to see the logs you must go to the functest.log or others
08:46:03 <jose_lausuch> #info question is: how difficult is to implement a flag in runtest.py to enable that output? does it make sense? is it useful for our users?
08:46:21 <morgan_orange> #info this is fine for CI or short test when done manually but it may be misleady for long tests (everything seems frozen)
08:46:52 <jose_lausuch> in my opinion, it is useful and we will get some questions in the mailing list about it (I already got some)
08:47:43 <ollivier> I'm afraid we can't use the config anymore
08:47:43 <morgan_orange> I like the -v option
08:48:11 <boucherv> I like the -v option too
08:48:26 <jose_lausuch> ollivier: I thought about a very ugly idea, and I bet you don't like it :)   what about if that flag is passed to run_tests.py, it will modify logging.ini and change it back at the end?
08:48:35 <ollivier> It will oblige core modules to init the loggers... I'm sorry I strongly focused on Alpine and I did'nt progress on that point.
08:49:26 <ollivier> jose_lausuch: The only good way is to adapt on the fly the objects described in logging.ini... But I think we can't
08:49:27 <jose_lausuch> ollivier: but do you have a good idea? maybe someone else can help to implememt it
08:49:33 <boucherv> How about a env variable for CI or not like check if a build_tag exist
08:50:22 <boucherv> if exist functest test are in wconsole and if not exist in console
08:50:32 <boucherv> and console by default
08:50:41 <jose_lausuch> boucherv: you mean, configuring logging.ini in prepare_env according to that env variable?
08:51:03 <ollivier> looks strenge but I see your point
08:51:52 <ollivier> why not. but the hack must be located in every entry point.
08:52:25 <jose_lausuch> really?
08:52:34 <jose_lausuch> why can't prepare_env do that modification once?
08:53:22 <ollivier> For classical functest processing it's fine...
08:53:48 <jose_lausuch> it's maybe a short term solution
08:53:51 <jose_lausuch> but not sure what's the best way
08:53:58 <jose_lausuch> boucherv: do you want to take the AP?
08:54:02 <ollivier> yes. I think it's the best short term  proposal I have seen
08:54:27 <ollivier> prepare_env hacks logging.ini according to options.
08:54:35 <boucherv> I have no time to do that :/
08:54:45 <jose_lausuch> I can try
08:54:45 <boucherv> Focus on my internship report ..
08:55:04 <jose_lausuch> but don't promise, also due to time, but will try, and it's not urgent
08:55:04 <ollivier> push the jira report inside it :)
08:55:32 <jose_lausuch> #action jose_lausuch propose a way to hack logging.ini in prepare_env according to env variable
08:55:36 <jose_lausuch> env variable CI_DEBUG ?
08:55:43 <jose_lausuch> since we already have it in the jjobs?
08:58:44 <jose_lausuch> ok
08:58:47 <jose_lausuch> I will use CI_DEBUG
08:58:57 <jose_lausuch> #topic Rest API
08:59:00 <jose_lausuch> morgan_orange: ?
08:59:57 <morgan_orange> #info restart needed this morning - the statbility issue reported by Serena is not fixed
09:00:19 <morgan_orange> #info simple restart is enough but I have a doubt on the version currently running as paginations per cases is no more there
09:00:42 <morgan_orange> #action serena-zte morgan_orange jose_lausuch sync on test API issues
09:01:10 <morgan_orange> note there is a sync meeting with Bitergia on thursday
09:01:15 <jose_lausuch> ok
09:01:22 <morgan_orange> for the moment their data set is very small
09:01:30 <morgan_orange> I had a look yesterday in the mongo DB
09:01:56 <morgan_orange> singleNodeRepl:PRIMARY>           db.results.count({"project_name":"functest"})
09:01:56 <morgan_orange> 141564
09:02:11 <morgan_orange> singleNodeRepl:PRIMARY>           db.results.count({"project_name":"yardstick"})
09:02:11 <morgan_orange> 11400
09:02:25 <morgan_orange> db.results.count({"project_name":"functest","version":"danube"})
09:02:25 <morgan_orange> 27353
09:02:33 <morgan_orange> db.results.count({"scenario":"os-nosdn-nofeature-ha"})
09:02:34 <morgan_orange> 24124
09:02:38 <morgan_orange> db.results.count({"scenario":"os-odl_l2-nofeature-ha"})
09:02:39 <morgan_orange> 22006
09:03:02 <jose_lausuch> what mongodb?
09:03:10 <morgan_orange> The test DB
09:03:26 <morgan_orange> so we shall have much more than we they are showing  https://opnfv.biterg.io/goto/dd9aa660f1fa5d8dd63f7f455609aa80
09:04:07 <jose_lausuch> #info sync meeting with Bitergia on thursday
09:04:10 <morgan_orange> #info sync meeting with Bitergia on Thursday (testing working group)
09:04:14 <morgan_orange> #undo
09:04:14 <collabot`> Removing item from minutes: <MeetBot.ircmeeting.items.Info object at 0x26e4e10>
09:04:36 <jose_lausuch> :)
09:04:41 <jose_lausuch> ok
09:04:50 <jose_lausuch> we can raise that topic on thursday
09:04:55 <jose_lausuch> any other thing?
09:04:56 <jose_lausuch> or we close?
09:05:09 <jose_lausuch> we didn't talk about danube.3.0 but you sent a very detailed email about the status
09:05:10 <jose_lausuch> thanks morgan_orange
09:07:00 <jose_lausuch> #endmeeting