04:59:39 #startmeeting QTIP Milesone C discussion 04:59:39 Meeting started Tue Sep 22 04:59:39 2015 UTC. The chair is VikramDham. Information about MeetBot at http://wiki.debian.org/MeetBot. 04:59:39 Useful Commands: #action #agreed #help #info #idea #link #topic. 04:59:39 The meeting name has been set to 'qtip_milesone_c_discussion' 04:59:49 Hello Everyone 05:00:03 Hi Virkam 05:00:39 Hello Vikram 05:01:04 #chair nahad 05:01:04 Current chairs: VikramDham nahad 05:02:00 As part of our preperation for milestone C, we have started listing EPICs, Stories and tasks within JIRA for QTIP 05:02:19 #topic roll call 05:02:35 #info Nauman 05:02:43 #info Mofassir 05:02:50 #info vikram 05:04:26 should we wait for Patrick & Iben for 2 minutes? 05:04:54 Debra clarified last week that as part of milestone C, the JIRA tasks should be detailed for those issues being covered for the first Sprint. The rest of the issues should be mentioned at a higher level by milestone C. 05:05:07 I’m here 05:05:07 I think iben was here. 05:05:17 #chair VikramDham nahad 05:05:17 Current chairs: VikramDham nahad 05:05:22 it’s after 10pm so will jsut be lurking 05:05:25 #topic roll call 05:05:33 ok 05:06:15 #link https://jira.opnfv.org/browse/QTIP 05:06:24 Nauman how many epics do we plan to cover in Sprint 1 05:07:08 #link: https://jira.opnfv.org/browse/QTIP-9 05:07:27 QTIP-9 EPIC. It deals with the framwork for QTIP and would be the core component 05:07:54 #link: https://jira.opnfv.org/browse/QTIP-20 05:08:13 QTIP-20 EPIC: It deals with compute testcases for QTIP 05:08:39 we will not be able to cover all the testcases for Sprint1 05:08:44 but 05:09:26 QTIP 34,37 05:09:32 for QTIP-9 - do you have schema for config.yaml? 05:10:23 Yes we have 05:10:56 Is it possible to add schema file to JIRA task? 05:10:58 you could find it at the bottom of the ehterpad: 05:11:05 ok 05:11:36 #action Add schema file to JIRA QTIP-16 05:13:31 Do we have template for output report? 05:13:43 or JIRA task to capture it? 05:14:07 Its not included in the first sprint 05:14:24 #info patrick 05:14:32 sorry for late 05:14:33 Hi Patrick 05:15:09 I think we are trying to work on the main framework for QTIP. We can only come to the result part once we are getting the data from the testcases 05:15:18 For the first sprint that is 05:15:44 ok that is fine 05:16:12 Regarding config.yaml - do we need different config.yaml for compute/network/storage ? 05:16:39 Yes. For each testcase there will be a different config.yaml file 05:17:01 How can the user create a suite using testcases? 05:17:25 What do you mean by a suite 05:17:50 The user will have the option of adding testcases by making their own config.yaml file 05:18:11 QTIP would be able to accept this config.yaml file 05:18:20 when user runs qtip would it pick all testcases to execute or does it read another file to identify the testcases? 05:18:49 the user specifices the benchmark from the command line 05:18:50 Or do we pass them all on qtip command line? 05:19:54 https://jira.opnfv.org/browse/QTIP-31 05:20:19 qtip --suite=compute --testcase=dhrystone I think this is fine for common users 05:20:51 Sure. Agree with Patrick 05:21:43 If the user wants to run both compute and networking cases, what will be the command? 05:22:25 Are we having the option of running multiple testcases at the same time? 05:22:33 As we discussed earlier in a previous meeting 05:22:48 if some one needs to run another benchmark, they should open another shell 05:23:04 but would someone want to run the network and compute testcases at the same time? 05:23:26 This is just added complexity 05:23:31 may not be same but serial possible? 05:23:41 Yes 05:23:43 yes 05:24:33 what will be command if we want qtip to run compute,networking suite or categories 05:25:03 qtip --suite=compute --testcase=all 05:25:21 qtip --suite=network --tescase=all 05:26:06 is qtip --suite=compute,network --testcase=dhrystone,pvp a possibility? 05:26:43 where pvp ill be some networking testcase 05:27:19 maybe we can support the two methods. qtip --suite=compute and qitp --test_list_file=test_case_list_files.md 05:27:47 we can read a md file to get the test cases list 05:28:22 I agree with Patrick. 05:28:39 we have listed it down but its a yaml file 05:28:41 oh ok 05:28:43 sorry i misunderstood 05:29:38 we can investigate into this 05:29:40 would the testcases be grouped into suites in the md file? 05:29:49 no 05:30:36 i think the test cases is independent from each other? 05:30:43 yes 05:30:44 grouping across suites is not required, because the config files will contain the necessary information for that testcase 05:30:50 yes but testcases are part of one of the group 05:31:36 but we are using the group categorization incase we need to run all the testcases for a certain category 05:31:48 like qtip --suite compute --test_case all 05:31:56 other than this , for each single test case 05:32:07 so the group I think is just for the simplifying the command input. 05:32:09 we are callings its config.yaml file 05:32:32 Exactly 05:32:33 won't the group help simplify presentation of output - 05:32:54 perhaps metrics such as QTIP compute, QTIP storage, QTIP network 05:33:10 That depends what the template of the Qtip's output. 05:33:18 all three numbers generated from the testcases in their group 05:33:33 initially we need 05:33:58 would be good to have json output of these QTIP numbers 05:34:02 yes 05:34:09 for each testcase / benchmark that is 05:34:10 and combine them for one QTIP number 05:34:24 one possbility is we run QTIP on a reference POD 05:35:28 but i am not sure whether we should output each test case in one json or we should output all the tase cases for sepcial categories(e,g, network) in one json. 05:36:20 Each testcase would generate its own results which would be in a different format Dhrystone would output txt files in a differetn format than DPI for example 05:36:36 we were thinking of having a consistent JSON format for each testcase first 05:37:00 after we have it we could combine them into a "category" wise JSON if needed 05:37:30 but we would need to convert the result for each testcase to JSON otherwise the results would be very fragmanted 05:37:50 ok. I think that is the detail about the output. we can investigate further. 05:37:58 Yes. 05:38:13 We are mostly interested in work for Sprint1 05:38:20 it inclides QTIP-9 issue 05:38:29 ok - so do we have final command format for qtip? 05:39:19 As we earlier discussed: qtip --suite compute test_case=dhrystone 05:39:35 qtip -suite network test_case=all 05:39:51 qtip --file 05:40:08 and lastly a list md file. We need to look a bit more into it 05:41:02 i think that's enough for initial development. 05:41:12 ok Nauman - can you please add these to JIRA 30? 05:41:18 ok 05:41:37 #link : https://jira.opnfv.org/browse/QTIP-18 05:41:42 just need to modify 05:42:01 ok 05:42:27 Changed: https://jira.opnfv.org/browse/QTIP-31 05:42:46 thanks! 05:43:43 Do we have all stories/tasks for Sprint 1 in JIRA or do you plan to add anymore? 05:43:53 No we have them in there 05:44:07 We now plan to finalize timelines 05:44:21 and later we could leverage the Agile Boards at JIRA 05:44:31 to plan the sprint for these tasks 05:44:35 ok 05:44:44 there were these emails circulating last night 05:44:57 I see Nauman you have provided some estimates for time - 05:45:03 yes i have 05:45:15 we plan to be done with it by Mid October 05:45:22 18th would be really great 05:45:30 as it allows some buffer time 05:46:09 So the emails circulating were listing tools on how to use JIRA agile boards for sprints 05:46:12 i will be looking into them for today 05:47:20 ok 05:47:37 Patrick would you like to add any stories/tasks to Sprint1? 05:48:05 Iben - would you like to add any stories/tasks to Sprint1? 05:49:07 ok - so we will go with Framwework + Compute testcases for Sprint1 05:49:29 +1 05:49:34 +1 05:50:47 All set 05:50:47 +1 05:51:17 ok we have Sprint1 scoped 05:51:34 for deadline I would suggest we target Oct 12th 05:51:53 as we expect big chunk of work in Sprint2 05:52:01 Thats a bit too early though. We would like to have a bufer zone 05:52:01 but the thing here is that 05:52:09 SPrint 1 would form the core of QTIP 05:52:24 It would provide the framework and the some of the compute testcases 05:52:32 the rest of the sprints would involve either: 05:52:37 1. Addind testcases 05:52:48 2. Generating results/output/ format 05:53:05 THis is why i beleive the first Sprint is of critical imprortance 05:53:42 this why we feel it would be great if we could have till 18th of October 05:54:26 i agree with Nauman, Sprint 1 will be the basis of QTIP 05:54:27 I would like us to be code complete by Nov 24th 05:54:42 as after that things are really slow during vacation time 05:55:46 And moreover you are already completing first iteration of core framework 05:56:01 Can we agree for the conclusion of Sprin1 on Thursday meeting? 05:56:10 date for Sprint1*? 05:56:40 ok 05:56:47 Thanks! 05:57:01 Thanks 05:57:05 you can reach out to QTIP team for everyone's input 05:57:37 Ok 05:57:52 Do we need to discuss anything else for today' meeting? 05:58:07 No. I don't think so. 05:58:30 I will circulate the QTIP JIRA issues for Sprint1 on the mailinging list 05:58:39 thanks Nauman 05:59:15 nd please ask everyone to review and send feedback by eod PST 9/24 06:00:15 thanks Patrick, Iben, Mofassir, QiLiang, mgkwill 06:00:22 Thanks everyone! 06:00:23 Thanks Vikram 06:00:29 Thanks everyone 06:00:39 #endmeeting