04:59:39 <VikramDham> #startmeeting QTIP Milesone C discussion
04:59:39 <collabot> Meeting started Tue Sep 22 04:59:39 2015 UTC.  The chair is VikramDham. Information about MeetBot at http://wiki.debian.org/MeetBot.
04:59:39 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
04:59:39 <collabot> The meeting name has been set to 'qtip_milesone_c_discussion'
04:59:49 <VikramDham> Hello Everyone
05:00:03 <nahad> Hi Virkam
05:00:39 <Mofassir> Hello Vikram
05:01:04 <VikramDham> #chair nahad
05:01:04 <collabot> Current chairs: VikramDham nahad
05:02:00 <nahad> As part of our preperation for milestone C, we have started listing EPICs, Stories and tasks within JIRA for QTIP
05:02:19 <VikramDham> #topic roll call
05:02:35 <nahad> #info Nauman
05:02:43 <Mofassir> #info Mofassir
05:02:50 <VikramDham> #info vikram
05:04:26 <VikramDham> should we wait for Patrick & Iben for 2 minutes?
05:04:54 <nahad> Debra clarified last week that as part of milestone C, the JIRA tasks should be detailed for those issues being covered for the first Sprint. The rest of the issues should be mentioned at a higher level by milestone C.
05:05:07 <iben> I’m here
05:05:07 <nahad> I think iben was here.
05:05:17 <VikramDham> #chair VikramDham nahad
05:05:17 <collabot> Current chairs: VikramDham nahad
05:05:22 <iben> it’s after 10pm so will jsut be lurking
05:05:25 <VikramDham> #topic roll call
05:05:33 <VikramDham> ok
05:06:15 <nahad> #link https://jira.opnfv.org/browse/QTIP
05:06:24 <VikramDham> Nauman how many epics do we plan to cover in Sprint 1
05:07:08 <nahad> #link: https://jira.opnfv.org/browse/QTIP-9
05:07:27 <nahad> QTIP-9 EPIC. It deals with the framwork for QTIP and would be the core component
05:07:54 <nahad> #link: https://jira.opnfv.org/browse/QTIP-20
05:08:13 <nahad> QTIP-20 EPIC: It deals with compute testcases for QTIP
05:08:39 <nahad> we will not be able to cover all the testcases for Sprint1
05:08:44 <nahad> but
05:09:26 <nahad> QTIP 34,37
05:09:32 <VikramDham> for QTIP-9 - do you have schema for config.yaml?
05:10:23 <nahad> Yes we have
05:10:56 <VikramDham> Is it possible to add schema file to JIRA task?
05:10:58 <nahad> you could find it at the bottom of the ehterpad:
05:11:05 <nahad> ok
05:11:36 <nahad> #action Add schema file to JIRA QTIP-16
05:13:31 <VikramDham> Do we have template for output report?
05:13:43 <VikramDham> or JIRA task to capture it?
05:14:07 <nahad> Its not included in the first sprint
05:14:24 <patrick1> #info patrick
05:14:32 <patrick1> sorry for late
05:14:33 <VikramDham> Hi Patrick
05:15:09 <nahad> I think we are trying to work on the main framework for QTIP. We can only come to the result part once we are getting the data from the testcases
05:15:18 <nahad> For the first sprint that is
05:15:44 <VikramDham> ok that is fine
05:16:12 <VikramDham> Regarding config.yaml - do we need different config.yaml for compute/network/storage ?
05:16:39 <nahad> Yes. For each testcase there will be a different config.yaml file
05:17:01 <VikramDham> How can the user create a suite using testcases?
05:17:25 <nahad> What do you mean by a suite
05:17:50 <nahad> The user will have the option of  adding testcases by making their own config.yaml file
05:18:11 <nahad> QTIP would be able to accept this config.yaml file
05:18:20 <VikramDham> when user runs qtip would it pick all testcases to execute or does it read another file to identify the testcases?
05:18:49 <Mofassir> the user specifices the benchmark from the command line
05:18:50 <VikramDham> Or do we pass them all on qtip command line?
05:19:54 <nahad> https://jira.opnfv.org/browse/QTIP-31
05:20:19 <patrick1> qtip --suite=compute --testcase=dhrystone  I think this is fine for common users
05:20:51 <nahad> Sure. Agree with Patrick
05:21:43 <VikramDham> If the user wants to run both compute and networking cases, what will be the command?
05:22:25 <nahad> Are we having the option of running multiple testcases at the same time?
05:22:33 <nahad> As we discussed earlier in a previous meeting
05:22:48 <nahad> if some one needs to run another benchmark, they should open another shell
05:23:04 <nahad> but would someone want to run the network and compute testcases at the same time?
05:23:26 <nahad> This is just added complexity
05:23:31 <VikramDham> may not be same but serial possible?
05:23:41 <nahad> Yes
05:23:43 <Mofassir> yes
05:24:33 <VikramDham> what will be command if we want qtip to run compute,networking suite or categories
05:25:03 <nahad> qtip --suite=compute --testcase=all
05:25:21 <nahad> qtip --suite=network --tescase=all
05:26:06 <VikramDham> is qtip --suite=compute,network --testcase=dhrystone,pvp a possibility?
05:26:43 <VikramDham> where pvp ill be some networking testcase
05:27:19 <patrick1> maybe we can support the two methods.   qtip --suite=compute    and  qitp --test_list_file=test_case_list_files.md
05:27:47 <patrick1> we can read a md file to get the test cases list
05:28:22 <nahad> I agree with Patrick.
05:28:39 <nahad> we have listed it down but its a yaml file
05:28:41 <nahad> oh ok
05:28:43 <nahad> sorry i misunderstood
05:29:38 <nahad> we can investigate into this
05:29:40 <VikramDham> would the testcases be grouped into suites in the md file?
05:29:49 <patrick1> no
05:30:36 <patrick1> i think the test cases is independent from each other?
05:30:43 <nahad> yes
05:30:44 <Mofassir> grouping across suites is not required, because the config files will contain the necessary  information for that testcase
05:30:50 <VikramDham> yes but testcases are part of one of the group
05:31:36 <nahad> but we are using the group categorization incase we need to run all the testcases for a certain category
05:31:48 <nahad> like qtip --suite compute --test_case all
05:31:56 <nahad> other than this , for each single test case
05:32:07 <patrick1> so the group I think is just for the simplifying  the command input.
05:32:09 <nahad> we are callings its config.yaml file
05:32:32 <nahad> Exactly
05:32:33 <VikramDham> won't the group help simplify presentation of output -
05:32:54 <VikramDham> perhaps metrics such as QTIP compute, QTIP storage, QTIP network
05:33:10 <patrick1> That depends what the template of the Qtip's output.
05:33:18 <VikramDham> all three numbers generated from the testcases in their group
05:33:33 <VikramDham> initially we need
05:33:58 <VikramDham> would be good to have json output of these QTIP numbers
05:34:02 <nahad> yes
05:34:09 <nahad> for each testcase / benchmark that is
05:34:10 <VikramDham> and combine them for one QTIP number
05:34:24 <VikramDham> one possbility is we run QTIP on a reference POD
05:35:28 <patrick1> but i am not sure whether we should output each test case in one json or we should output all the tase cases for sepcial categories(e,g, network) in one json.
05:36:20 <nahad> Each testcase would generate its own results which would be in a different format Dhrystone would output txt files in a differetn format than DPI for example
05:36:36 <nahad> we were thinking of having a consistent JSON format for each testcase first
05:37:00 <nahad> after we have it we could combine them into a "category" wise JSON if needed
05:37:30 <nahad> but we would need to convert the result for each testcase to JSON otherwise the results would be very fragmanted
05:37:50 <patrick1> ok. I think that is the detail about the output. we can investigate further.
05:37:58 <nahad> Yes.
05:38:13 <nahad> We are mostly interested in work for Sprint1
05:38:20 <nahad> it inclides QTIP-9 issue
05:38:29 <VikramDham> ok - so do we have final command format for qtip?
05:39:19 <nahad> As we earlier discussed: qtip --suite compute test_case=dhrystone
05:39:35 <nahad> qtip -suite network test_case=all
05:39:51 <nahad> qtip --file <to your testcase config.yaml file>
05:40:08 <nahad> and lastly a list md file. We need to look a bit more into it
05:41:02 <patrick1> i think that's enough for initial development.
05:41:12 <VikramDham> ok Nauman - can you please add these to JIRA 30?
05:41:18 <nahad> ok
05:41:37 <nahad> #link : https://jira.opnfv.org/browse/QTIP-18
05:41:42 <nahad> just need to modify
05:42:01 <VikramDham> ok
05:42:27 <nahad> Changed: https://jira.opnfv.org/browse/QTIP-31
05:42:46 <VikramDham> thanks!
05:43:43 <VikramDham> Do we have all stories/tasks for Sprint 1 in JIRA or do you plan to add anymore?
05:43:53 <nahad> No we have them in there
05:44:07 <nahad> We now plan to finalize timelines
05:44:21 <nahad> and later we could leverage the Agile Boards at JIRA
05:44:31 <nahad> to plan the sprint for these tasks
05:44:35 <VikramDham> ok
05:44:44 <nahad> there were these emails circulating last night
05:44:57 <VikramDham> I see Nauman you have provided some estimates for time -
05:45:03 <nahad> yes i have
05:45:15 <nahad> we plan to be done with it by Mid October
05:45:22 <nahad> 18th would be really great
05:45:30 <nahad> as it allows some buffer time
05:46:09 <nahad> So the emails circulating were listing tools on how to use JIRA agile boards for sprints
05:46:12 <nahad> i will be looking into them for today
05:47:20 <VikramDham> ok
05:47:37 <VikramDham> Patrick would you like to add any stories/tasks to Sprint1?
05:48:05 <VikramDham> Iben - would you like to add any stories/tasks to Sprint1?
05:49:07 <VikramDham> ok - so we will go with Framwework + Compute testcases for Sprint1
05:49:29 <nahad> +1
05:49:34 <patrick1> +1
05:50:47 <iben_> All set
05:50:47 <Mofassir> +1
05:51:17 <VikramDham> ok we have Sprint1 scoped
05:51:34 <VikramDham> for deadline I would suggest we target Oct 12th
05:51:53 <VikramDham> as we expect big chunk of work in Sprint2
05:52:01 <nahad> Thats a bit too early though. We would like to have a bufer zone
05:52:01 <nahad> but the thing here is that
05:52:09 <nahad> SPrint 1 would form the core of QTIP
05:52:24 <nahad> It would provide the framework and the some of the compute testcases
05:52:32 <nahad> the rest of the sprints would involve either:
05:52:37 <nahad> 1. Addind testcases
05:52:48 <nahad> 2. Generating results/output/ format
05:53:05 <nahad> THis is why i beleive the first Sprint is of critical imprortance
05:53:42 <nahad> this why we feel it would be great if we could have till 18th of October
05:54:26 <Mofassir> i agree with Nauman, Sprint 1 will be the basis of QTIP
05:54:27 <VikramDham> I would like us to be code complete by Nov 24th
05:54:42 <VikramDham> as after that things are really slow during vacation time
05:55:46 <VikramDham> And moreover you are already completing first iteration of core framework
05:56:01 <nahad> Can we agree for the conclusion of Sprin1 on Thursday meeting?
05:56:10 <nahad> date for Sprint1*?
05:56:40 <VikramDham> ok
05:56:47 <nahad> Thanks!
05:57:01 <Mofassir> Thanks
05:57:05 <VikramDham> you can reach out to QTIP team for everyone's input
05:57:37 <nahad> Ok
05:57:52 <VikramDham> Do we need to discuss anything else for today' meeting?
05:58:07 <nahad> No. I don't think so.
05:58:30 <nahad> I will circulate the QTIP JIRA issues for Sprint1 on the mailinging list
05:58:39 <VikramDham> thanks Nauman
05:59:15 <VikramDham> nd please ask everyone to review and send feedback by eod PST 9/24
06:00:15 <VikramDham> thanks Patrick, Iben, Mofassir, QiLiang, mgkwill
06:00:22 <nahad> Thanks everyone!
06:00:23 <Mofassir> Thanks Vikram
06:00:29 <Mofassir> Thanks everyone
06:00:39 <VikramDham> #endmeeting