08:00:46 <jose_lausuch> #startmeeting Functest weekly meeting April 11th 2017
08:00:46 <collabot`> Meeting started Tue Apr 11 08:00:46 2017 UTC.  The chair is jose_lausuch. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:00:46 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:00:46 <collabot`> The meeting name has been set to 'functest_weekly_meeting_april_11th_2017'
08:00:54 <jose_lausuch> #topic role call
08:00:58 <jose_lausuch> #info Jose Lausuch
08:01:00 <viktor_t> #info Viktor Tikkanen
08:01:10 <SerenaFeng> #info SerenaFeng
08:01:12 <Shuya_OOL> #info Shuya(OOL)
08:01:16 <jose_lausuch> #info agenda: https://wiki.opnfv.org/display/functest/Functest+Meeting#FunctestMeeting-11/04(8UTC)
08:01:18 <hideyasu> #info Hideyasu (OOL)
08:01:29 <ollivier> #info Cédric
08:01:29 <jose_lausuch> viktor_t: long time no see!!
08:01:36 <LindaWang> #info Linda Wang
08:01:44 <HelenYao> #info Helen Yao
08:02:31 <jose_lausuch> #topic Action point follow-up
08:03:06 <jose_lausuch> I see I got several APs :)
08:03:13 <jose_lausuch> didn't have time to work on them... sorry
08:03:18 <jose_lausuch> #action jose_lausuch plan a slot for this topic during the Hackfest in Paris
08:03:31 <jose_lausuch> #action jose_lausuch propose demo for the hackfest about the available tooling in releng
08:03:38 <jose_lausuch> #action jose_lausuch review https://gerrit.opnfv.org/gerrit/#/c/32083/ and associate patches (32083 impacting all the feature projects..need to be merged to avoid too many rebasing)
08:04:01 <ollivier> And https://gerrit.opnfv.org/gerrit/#/c/33195/ too
08:04:19 <jose_lausuch> I think I will have time to look at gerrit today
08:04:36 <jose_lausuch> SerenaFeng plans a short slide deck for the meeting planned on the 12th (could be reused for the plugfest)
08:04:48 <jose_lausuch> SerenaFeng: what meeting?
08:04:55 <ollivier> Any testing result is welcome as our code is not fully covered
08:04:56 <SerenaFeng> I don't know
08:05:08 <SerenaFeng> ollivier do you know that?
08:05:21 <ollivier> ?
08:05:33 <SerenaFeng> I didn't show up on last weekly meeting due to QingMing Vocation in China
08:05:49 <ollivier> I know we have to speak together about status codes...
08:06:01 <SerenaFeng> ollivier: SerenaFeng plans a short slide deck for the meeting planned on the 12th (could be reused for the plugfest)
08:06:03 <SerenaFeng> this one
08:06:13 <ollivier> I don't remember anything about a meeting...
08:06:23 <jose_lausuch> ok
08:06:27 <SerenaFeng> ok
08:06:30 <jose_lausuch> I don't see anything in the minutes...
08:06:48 <jose_lausuch> ok, anyway
08:07:05 <jose_lausuch> all complete Functest postmortem (what was ++, +, - or --)
08:07:11 <jose_lausuch> I haven't had a look yet
08:07:14 <jose_lausuch> #action all complete Functest postmortem (what was ++, +, - or --)
08:07:44 <jose_lausuch> #link https://etherpad.opnfv.org/p/FunctestDanubePostMortem
08:08:11 <jose_lausuch> ALL review Functest Euphrates page
08:08:19 <jose_lausuch> I think there are some additions in that page
08:08:24 <jose_lausuch> I've seen some emails around
08:08:29 <jose_lausuch> we have to talk about it now anyway
08:09:35 <jose_lausuch> what do you want to do guys, to spend some time in the issues we have or to talk about euphrates?
08:09:57 <ollivier> we could aso review https://wiki.opnfv.org/display/functest/Review+checkpoints
08:10:23 <jose_lausuch> yes
08:10:41 <jose_lausuch> then, let's do it
08:10:48 <jose_lausuch> #topic Review checkpoints
08:10:55 <jose_lausuch> #link https://wiki.opnfv.org/display/functest/Review+checkpoints
08:11:51 <jose_lausuch> do we all agree on the content of this wiki?
08:12:26 <HelenYao> For this line -> the review must not break the unit test results or hugely decrease the coverage trend (-2%), how to calculate the 2%? There is a patch by Ashish which is setting the line decreasing as 10
08:12:51 <HelenYao> https://gerrit.opnfv.org/gerrit/#/c/32763/
08:13:11 <ollivier> 2% or 10% is ok. But we have to list all functest modules
08:13:14 <HelenYao> if we all agree with the -2%，the patch needs to be changed
08:13:37 <HelenYao> it's 10 lines, not 10%
08:14:09 <ollivier> The most important point is "a review fixing a bug should add the related unit tests"
08:14:20 <HelenYao> yes. I totally agree
08:14:34 <SerenaFeng> will -2% be detected by jenkins?
08:14:50 <SerenaFeng> if so I agree
08:14:52 <ollivier> Maybe 2% is too short to vote -1
08:14:58 <HelenYao> if the number is not fixed, I would put it as e.g. -2%
08:15:35 <SerenaFeng> I think the point is jenkins can provide that decrease threshold for us
08:15:44 <HelenYao> SerenaFeng: there is some changed needed for jenkins to detect the line change with the help of Ashish's patch
08:15:46 <SerenaFeng> it is difficult to do it manually
08:15:53 <ollivier> But 2% is too much if the code was previously fully covered 100%...
08:16:02 <jose_lausuch> if it's 2% or 10% it doesn't matter, I think, no need to be strick to those numbers, the important thing is that it doesn't decrease the coverage hugely...
08:16:22 <SerenaFeng> jose_lausuch, I agree
08:16:35 <jose_lausuch> how can we detect exactly what is the coverage decrease of a certain patch?
08:16:35 <ollivier> We could list all the modules (even if not covered) and set to 10% and let see
08:16:38 <HelenYao> if we want to add the CI gating, the number is ought to be decided in the near future
08:16:43 <SerenaFeng> the number itself is doesn't matter
08:17:14 <ollivier> agree. The most important is adding unit test for every bug and new features...
08:17:20 <SerenaFeng> agree
08:17:24 <jose_lausuch> yep
08:17:32 <HelenYao> i think it's a two-step decision, 1. agree with the CI gating 2. agree with the coverage rate
08:17:42 <jose_lausuch> remove the -2% to avoid confusion?
08:17:45 <ollivier> 1/ Agree 2/  10?
08:17:46 <HelenYao> i totally agree with the gating
08:18:46 <ollivier> But we have to update the review to list all functest modules and to check why several are not covered at all (as features).
08:19:21 <jose_lausuch> ollivier: you mean in the jjob ?
08:19:29 <ollivier> no by PTL
08:19:30 <SerenaFeng> because unittest is not required at that time
08:19:37 <ollivier> agree.
08:19:53 <SerenaFeng> so there's no unit test covered for feature
08:20:05 <ollivier> But we have to know where functest could fail.
08:21:17 <jose_lausuch> besides features, is there any other module which is not covered?
08:21:21 <ollivier> I don't care about the past mistakes. I think we must now cover the remaining part.
08:22:02 <ollivier> For features, it was quite simple to cover and I could have helped if I knew them.
08:22:31 <HelenYao> the reason why there is no unit test for feature is, the code is simply __init__
08:22:32 <ollivier> run_unit_tests.sh must list all the functest modules
08:23:11 <ollivier> Yes. But Serena raised a bug on my review which could be easily covered by UT.
08:23:27 <ollivier> Just be testing constructors.
08:23:30 <ollivier> by
08:23:54 <jose_lausuch> ok, no blame to anyone
08:24:01 <jose_lausuch> let's cover everything from now on
08:24:04 <SerenaFeng> ok,my apology for not adding unittest
08:24:08 <ollivier> no it's not.
08:24:24 <ollivier> That's not the purpose of my sentences.
08:25:00 <ollivier> We have to know where we should add unit tests. That's my point.
08:25:12 <jose_lausuch> I agree
08:25:14 <jose_lausuch> so
08:25:15 <jose_lausuch> besides features
08:25:16 <ollivier> Even the simple unit tests are relevant
08:25:18 <jose_lausuch> is there any thing?
08:25:22 <jose_lausuch> have you detected any other area?
08:25:50 <ollivier> No.
08:25:56 <HelenYao> constants.py
08:26:09 <ollivier> SerenaFeng: Don't worry. Thank you to raise the constructor issues.
08:27:20 <SerenaFeng> ollivier: agree we cannot ignore unit test because it is simple
08:27:31 <jose_lausuch> ok
08:27:35 <jose_lausuch> going back to the review checkpoints
08:27:46 <jose_lausuch> unix permissions
08:27:49 <jose_lausuch> I think that's clear
08:27:55 <HelenYao> recalling my last comment - I just checked the constants.py and it's 100% covered
08:27:59 <jose_lausuch> OPNFV Functest programming rules?
08:28:48 <ollivier> We mainly follow OpenStack rules. I will be great to run pylint too.
08:29:20 <jose_lausuch> ollivier: I think we can create a job for pylint
08:29:47 <SerenaFeng> if pylint is used, I high suggest to leverage jjb to do the check
08:29:48 <jose_lausuch> we use this for pep8 https://git.opnfv.org/releng/tree/jjb/releng/opnfv-lint.yml
08:29:52 <ollivier> Yes but it must be seen as an helper. No need to vote -1 if pylint sore is too low
08:29:58 <jose_lausuch> SerenaFeng: yes, that is the point
08:30:12 <SerenaFeng> manual work can not be guaranteed
08:30:15 <jose_lausuch> ollivier: you have run pylint over the whole functest code?
08:31:08 <ollivier> 10/10 on ODL, TestCase and Features (last reviews). I don't check everything.
08:31:30 <ollivier> It will be great if everyone uses it to increase the quality.
08:31:51 <ollivier> Some checks could be disabled as I did
08:32:24 <jose_lausuch> shall we take the AO to create a jjob?
08:32:37 <ollivier> AO? AP?
08:32:42 <jose_lausuch> AP sorry
08:33:14 <SerenaFeng> agree to create only after all modules in Functest have done it
08:33:23 <ollivier> Sure as coverage it could help.
08:33:36 <SerenaFeng> I'm afraid currently only ODL will pass
08:33:47 <jose_lausuch> yes, but if we don't vote -1 for that job it's ok
08:33:50 <ollivier> Again. It must be seen as an helper
08:34:26 <ollivier> Try to increase the score, that's all.
08:34:29 <HelenYao> +1
08:34:37 <jose_lausuch> I will ask opnfv helpdesk
08:34:39 <jose_lausuch> to do that
08:34:44 <HelenYao> for the commit msg, I thought the JIRA link is right after the first line of brief description. In the wiki, it says the JIRA is at the very end. How do you think?
08:35:04 <ollivier> For instance docstrings are required only for external (internal) API
08:35:24 <jose_lausuch> well, at the beginning I said the second line, but it really doesn't matter as long as you put it
08:35:30 <ollivier> JIRA must be at the end
08:35:42 <jose_lausuch> what's important is "subsequent lines should be wrapped at 72 characters"
08:35:52 <jose_lausuch> we should keep tidy commit messages
08:36:04 <jose_lausuch> let's agree on put JIRA at the end
08:36:12 <jose_lausuch> it's also according to openstack rules
08:36:34 <LindaWang> ollivier:  the hyperlink of " OpenStack Git Commit Good Practice" is not assesible for me.
08:36:53 <ollivier> True. I am fixing it
08:37:32 <ollivier> LindaWang: https://wiki.openstack.org/wiki/GitCommitMessages
08:37:41 <jose_lausuch> I would put an example of good commit message
08:39:03 <ollivier> please see "fold"
08:39:34 <jose_lausuch> fold?
08:40:47 <viktor_t> jose_lausuch: Agree, an example visualizes rules in a compact form
08:41:07 <ollivier> fold - wrap each input line to fit in specified width
08:41:13 <ollivier> (man fold)
08:41:28 <jose_lausuch> ah ok
08:41:36 <ollivier> fold -w 72 -s .git/COMMIT_EDITMSG
08:41:39 <jose_lausuch> ollivier: will you add an example?
08:41:48 <ollivier> jose_lausuch: yes
08:41:51 <jose_lausuch> thanks
08:42:00 <jose_lausuch> commit message section is clear
08:42:04 <jose_lausuch> programming guidelines
08:42:16 <jose_lausuch> #action ollivier add example of good commit message
08:42:17 <ollivier> LindaWang: fixed. Thank you
08:42:32 <LindaWang> :) i
08:42:54 <jose_lausuch> I agree with everything basically
08:43:06 <jose_lausuch> can you explain
08:43:06 <jose_lausuch> we prefer python methods (e.g. those defined in os library) to bash commands called via os.system()
08:43:47 <SerenaFeng> how about using subprocess?
08:44:02 <SerenaFeng> I think the information provided by os.system() is quite few
08:45:14 <ollivier> What's the purpose of calling cd in synstem
08:45:38 <ollivier> via os.system()?
08:45:55 <ollivier> or to parse output via grep.
08:47:07 <ollivier> They are python librairies designed for that.
08:47:07 <jose_lausuch> you mean what we were doing in features
08:47:11 <SerenaFeng> because some feature tests implemented in shell
08:47:18 <jose_lausuch> cd DIR && bash whatever
08:47:49 <SerenaFeng> and they require the scripts to be executed within that directory
08:47:55 <ollivier> I agree calling bash script offered by 3rd party. But changing dir via os.system is strenge
08:48:25 <ollivier> I think there are issues in utils too.
08:48:27 <jose_lausuch> we should call os. call with the appropriate directory
08:49:56 <ollivier> cd is just an exemple.
08:49:59 <jose_lausuch> more questions about this topic?
08:50:14 <SerenaFeng> and within os.system() it will not impact Functest itself
08:50:58 <ollivier> please see ./functest/opnfv_tests/sdn/onos/teston/adapters/foundation.py
08:51:06 <ollivier> cmd = "cat " + self.logfilepath + " | grep Fail"
08:51:29 <ollivier> about switching to a python way
08:52:03 <ollivier> output = os.popen('cat /root/.ssh/id_rsa.pub')
08:52:19 <jose_lausuch> we can probably work on that to make it more pythonized
08:52:30 <ollivier> Again I don't blame anyone. They are just examples...
08:53:41 <jose_lausuch> so
08:53:43 <jose_lausuch> unix permissions
08:53:47 <jose_lausuch> I think it's clear too
08:54:39 <SerenaFeng> again suggest to add jjb to detact it
08:55:15 <SerenaFeng> any restriction should be promised by jenkins
08:55:27 <jose_lausuch> to check the permissions?
08:55:46 <SerenaFeng> yes, if we want to make restriction on that
08:55:53 <SerenaFeng> manual work is not reliable
08:56:07 <jose_lausuch> yes, good idea
08:56:20 <ollivier> Agree. It's quite hard to detect unix permissions for the author or the reviewers.
08:56:27 <jose_lausuch> #action jose_lausuch propose 2 new gating jobs: pylint and unix permissions
08:56:52 <SerenaFeng> we can not submit patches to change unit permissions all the time
08:57:50 <ollivier> Jenkins could vote -1 for this point (as pep8), it will help the author as well
08:58:25 <jose_lausuch> yes
08:58:29 <jose_lausuch> ok
08:58:36 <jose_lausuch> we ran out of time, but we had good discussions
08:58:42 <jose_lausuch> what about voting -1 -2 ?
08:59:09 <SerenaFeng> I should say Functest is so harsh now, much more harsh than OpenStack
08:59:33 <ollivier> I don't think so :)
08:59:57 <HelenYao> so long as the rule is clear, I think members will obey the rules
09:00:09 <jose_lausuch> all the software projects go like this when they get mature
09:00:13 <jose_lausuch> and it's good
09:00:17 <jose_lausuch> to follow some guidelines
09:00:20 <jose_lausuch> to improve quality
09:00:32 <jose_lausuch> ollivier: will you add something about when to vote -1 and when to vote -2 ?
09:00:42 <ollivier> I did.
09:00:51 <HelenYao> adding examples for the rules will help the readers better understand the rules
09:00:57 <ollivier> I think -2 should be used when 2 core diagree.
09:01:00 <jose_lausuch> ollivier: sorry, didn't see it
09:01:28 <jose_lausuch> for example, if I see that something is really wrong, can I vote -2 ?
09:02:08 <ollivier> I would say yes if the change must be stopped as it's fully false.
09:02:31 <jose_lausuch> ok
09:02:35 <jose_lausuch> good
09:02:41 <ollivier> Normally you could vote -2 if the vote is already +2
09:02:50 <jose_lausuch> ok
09:02:51 <jose_lausuch> we will talk about Euphrates next week
09:02:57 <jose_lausuch> who is coming to the hackfest?
09:03:13 <ollivier> For this point I voted -1 and sometimes the change was merged anyway :)
09:03:28 <viktor_t> juhak is coming there
09:03:40 <jose_lausuch> ok
09:03:52 <jose_lausuch> SerenaFeng: I know you are coming
09:03:54 <jose_lausuch> HelenYao: ?
09:03:59 <SerenaFeng> yes
09:04:07 <HelenYao> no:(
09:04:37 <SerenaFeng> I am preparing for the TestAPI presentation
09:05:01 <jose_lausuch> HelenYao: ok, so we can still discuss our E-release plan next week so you can still give your input
09:05:08 <jose_lausuch> SerenaFeng: I will put a slot in the agenda
09:05:18 <HelenYao> that is great
09:05:25 <SerenaFeng> okey, that would be great
09:06:29 <jose_lausuch> good
09:06:30 <jose_lausuch> thanks
09:06:32 <jose_lausuch> #endmeeting