13:57:33 <[1]JonasB> #startmeeting BGS Fuel status meeting 13:57:33 Meeting started Thu May 28 13:57:33 2015 UTC. The chair is [1]JonasB. Information about MeetBot at http://wiki.debian.org/MeetBot. 13:57:33 Useful Commands: #action #agreed #help #info #idea #link #topic. 13:57:33 The meeting name has been set to 'bgs_fuel_status_meeting' 13:57:42 <[1]JonasB> #info Jonas Bjurel 13:57:47 #info Daniel Smith 13:58:11 parallel weekly functest meeting 13:58:14 #info Jose Lausuch 13:58:29 #info Michal Skalski 13:58:36 #info Szilard Cserey 13:59:55 <[1]JonasB> So I guess the ODL testing is the topic for today, but since Jose is in a parallell meeting maybe it is not worth while? 14:00:17 <[1]JonasB> Jose what do you think? 14:00:25 #info Stefan Berg 14:00:34 I can try 14:01:03 <[1]JonasB> Jose: a short summary? 14:01:10 difficult :) 14:01:23 its a bit of a messy situation right now 14:01:26 lets see 14:02:29 #info we are getting to know now how foreman implements ODL, so that we can do the same things for Fuel deployments 14:03:09 #info Foreman deployment currently ongoing, later we will check what is needed to understand how it is done 14:03:33 #info troubles about stablishing priorities 14:03:41 #info discussions ongoing 14:03:50 info from Chris (Same thing Jose and I asked Dan) - is that FOREMAN deployed Openstack with ODL config and then switches as well 14:04:14 #info focus for Fuel deployment: 14:04:15 still not clear when / where they are testing and how that is done (i.e. test Openstack and then do a switch to ODL mode and run the "same" tests again). 14:04:27 <[1]JonasB> Ok, Daniel do you have the credentials to log into POD 2 and fish the bridge config? 14:04:35 I have credentials 14:04:37 no.. i dont, i mean i can go looking 14:04:38 I can help on that 14:05:06 #info 1) Deploy 2) Run functes SET1 3) Switch to ODL + reboot (if nonHA deployment) 4) Run functest SET2 (if nonHA deployment) 14:05:08 but really - four days to hear - i can go looking myself? - not very collaborative.. where is the other people with Jose and i working on this for a week reelase? 14:05:21 $info SET1 = Tempest + vPing + Rally Bench, SET2 = ODL single test using robot that Peter wrote (creating neutron net/subnet/port and checking the result also in ODL rest api, similar to your curl approach) 14:05:35 thanks Jose - is that what is being done on FOREMAN then? 14:05:43 No 14:05:47 foreman currently does: 14:05:54 <[1]JonasB> jose: Can you help Daniel to get the config he wants to look at. 14:05:57 1) Deploy 2) Run functes SET1+SET2 14:06:02 yes 14:06:09 How?? that doesnt make sense 14:06:27 Dan just informed Chris that FOREMAN doesnt come up in ODL mode 14:06:31 so SET2 then always fails 14:06:40 unless they do the same switch that we are outlining for FUEL 14:06:44 yes, it fails and asked for help to troubleshoot 14:06:57 well, it was like this for Fuel as well, before knowing about that ODL impacts O/S tests 14:07:00 SET2 is ALWAYS GOING TO FAIL cause the opentack is not in ODL mode 14:07:01 eactly 14:07:06 <[1]JonasB> jose: what fails - POD2? 14:07:06 thats the info we are waiting for, or? 14:07:07 can someone merge this: https://gerrit.opnfv.org/gerrit/#/c/687/ 14:07:16 ODL test on POD2, I saw a failure yesterday 14:07:17 Found the line that was killing rst2pdf 14:07:20 (yes aric) 14:07:22 i will now 14:07:45 msg aricg done 14:07:49 thx 14:07:59 ok - so that its clear 14:08:06 POD2 is doing the same semantrics we are doing 14:08:11 your SET2 will alays fail 14:08:18 cause you havent set POD2 (FORMEAN) into ODL mode 14:08:24 #info need to modify the Jenkins job to trigger ODL test when it makes sense 14:08:27 they are doing (as i have come to piuck apart) 14:08:32 the same thing we are 14:08:36 ok 14:08:38 now - my questino still stands 14:08:43 when you put the system in ODL mode 14:08:48 are you only doing SET2 14:08:49 taht is the information we missed (I had no clue about) and we need to integrate 14:08:53 or are you doing all the tests again? 14:08:55 no 14:08:59 the idea would be 14:09:07 correct me if Im wrong or if it doesnt make sense 14:09:10 the same as Fuel 14:09:23 1) Deploy 2) Run functes SET1 3) Switch to ODL 4) Run functest SET2 14:09:31 <[1]JonasB> jose: sound as the right thing to do 14:09:38 ok 14:09:50 but still need info about the procedure of how to do step 3 14:10:01 <[1]JonasB> lmcdasm: Do you agree? 14:10:27 well.. if that is what we want to do then fine 14:10:33 shouldnt we be doing the same things on both systems 14:10:34 sorry for the ignorance, but I dont know about how ODL is integrated and switched on/off and this things 14:10:38 since they are both deploying the same way? 14:10:46 This needs to be rebased now, and it will pass as well. https://gerrit.opnfv.org/gerrit/#/c/673/ 14:10:51 that is what we have to figure out now 14:10:55 what makes sense 14:11:14 what would you say it makes sense Daniel? 14:11:18 i cant tell you 14:11:20 ChrisPriceAB: I guess that one was for you ^ :) 14:11:23 sorry.. i dont set the release spec 14:11:31 this is defined from the PTO and the TSC 14:11:57 we have asked for this infomation weekly (the whole picture) for about a month 14:12:07 and get little pieces.. so i think that the testing team shoudl sit down 14:12:15 <[1]JonasB> lmcdasm: I think we need to be pragmatic here and now and propose the way forward 14:12:16 and come up with the plan and ask the two deploers how they are doing things 14:12:19 wait 14:12:29 I got this info right now from Morgan (didnt know) 14:12:40 <[1]JonasB> lets hear 14:13:06 yesterday, 6 ODL tests were OK 12 failed 14:13:32 can you give a bit more context? 14:13:53 on which POD.. and since there are only 10 cases, im not sure how that can be (reading from Peters website). 14:14:17 ok, forget that, the errors are due to some other things 14:14:20 need to troubleshoot 14:14:25 sorry, forget what I have said 14:14:35 https://wiki.opnfv.org/r1_odl_suite 14:14:40 this is what i understand we are testing again 14:14:47 is that the complete test case for ODL? 14:15:09 yes 14:15:14 on the table it says that all 16 cases have passed for POD2 14:15:23 so there is a discrpancy.. 14:15:24 manually yes 14:15:27 from jenkins not yet 14:15:41 I havent done myself on POD2, I think Peter will know 14:15:50 ok.. so then the FUEL side (since you did it on Monday can be updated as well)( 14:15:51 ? 14:15:52 I managed to run it on POD1 manually succesfully as well 14:15:52 /window show 5 14:16:10 yes, but doing some tweaks 14:16:19 <[1]JonasB> jose: Can you then update the Wiki page 14:16:21 what tweeks? 14:16:33 bugs in the script which work manually but not from Jenkins 14:16:34 on it aricg, will report back 14:16:35 it should be the same manualy tests that peter ran no? 14:16:39 the goal is to make it ran from jenkins 14:16:51 yes, and I should tell him 14:17:03 that it is not acceptable to have it PASSED if its not like that from Jenkins 14:17:18 and currently they are not PASSED (at least not all of them) in last functest job last night 14:17:33 I wil update the wiki 14:17:34 sure - sounds good.. im still super confused that we dont have a solid test specifictaion 14:17:36 this is a bit old info 14:17:42 is it 12 cases, I count 16, 14:17:59 forget about the 12 14:18:02 and sure we can say " in Fuel - we are doing this", but then we arent comparing apples to apples if the setup for FOREMAN is way different 14:18:11 but seems like maybe im the only one screaming about this. 14:18:25 I am also screaming now :) 14:18:42 what do you suggest? 14:18:50 to have different steps for Fuel and Foreman? 14:19:07 well.. i think a slower approach is needed 14:19:15 first - are we clear how to set the FOREMAN machine into ODL mode? 14:19:26 second- we make the semantic in FUEL the same 14:19:34 for 1) Im not, who is? 14:19:45 aricg: ping 14:19:48 sorry 14:19:49 <[1]JonasB> jose: pbanzi must know 14:19:51 fdegir: ping 14:19:59 third - the testing team releases a clear ppt / or one pager that shows the tesing scope and how the test are run - just like the 5+ charts differenet people have given 14:20:11 hold on a second here 14:20:17 jose_lausuch: yes 14:20:22 can we have 2 json jobs, 1 for foreman and one for fuel if we have different approach about how to swithc to ODL? 14:20:27 how can the testing reports be coming in at all if the testing team doesnt know how to set it into ODL? 14:20:38 so that means really that no testing on ODL (from functest) has been done ? 14:21:19 jose_lausuch: you mean 2 different jobs - one for fuel and the other for foreman 14:21:19 we assumed that was transparent for the tests 14:21:23 yes fdegir 14:21:26 if so, yes 14:21:46 ok.. well. hmm. then i would suggest that we take a step back and get familliar with what we are testing. 14:21:58 the switching you are talking about is something new, we assumed ODL was available from scratch (wrong assumption, but nobody said the opposite) 14:22:03 it does no good to saY " i got a passed" if we dont understand (at least a bit) what we are actually testing 14:22:30 I think we need peter for this discussion 14:22:44 ODL implementation creates a overlay so changes all the networking - which is why im asking if you should not be doing the whole testing again - since the ODL tests now just show a simple subset, but we know that when you try to do other Openstack things, it breaks, 14:22:46 thanks fdegir 14:23:17 so then I would like to know exactly what you asked: - How are we deploying things (single deploy, double deploy, deploy then alter?). 14:23:21 anyway - its moot 14:23:28 <[1]JonasB> ping trozet 14:23:36 if the methods for how to change the mode to ODL arent in place in the testing machine yet 14:23:39 that is your first step 14:23:53 (after you read the overview section from the opendaylight site that is :) ) 14:23:59 hi [1]JonasB 14:24:26 I dont want to spend 1 day understanding that, I asked for help about who could answer taht quickly 14:24:32 we dont have time to build up competence right now 14:24:36 <[1]JonasB> trozet: Can you share som light on if ODL is automatically activated when you deploy Forman 14:24:44 after Arno we have to for sure 14:24:48 but we have to get things done 14:25:40 that doesnt make sense. 14:25:50 going back to the point of testing, if you are just looking for a greenlight 14:25:54 then how do you know its acutally working 14:26:36 if you dont take 10 minutes to read the page then it falls to others to take that time to give you the explanation.. anyway - doesnt matter, but the picture of what ODL is doing speaks a thousand words 14:26:51 and would have avoided the assumption that a) tthere is no difference between when the system is in ODL mode (vxlan) or not 14:27:06 [1]JonasB: ODL is configured if the odl_flag is set to true 14:27:07 b) undestand the IRP - integration reference points - so that the test cases have some relevance. 14:27:07 if its 10 minutes then its ok, I was expecting hundres of pages of documentation 14:27:16 in our yaml inventory file 14:27:37 hey Tim - so can you outline the steps that are needed to switcvh to ODL mode once the system is deployed? 14:27:43 <[1]JonasB> trozet: DO you know if that is default? 14:27:44 or do you only deploy one way or the other? 14:28:04 these are the key questions 14:28:11 [1]JonasB: it is the default 14:28:18 hi lmcdasm 14:28:29 so we are deploying every day with ODL mode 14:28:37 yes 14:28:40 and the vxlan tunnel are coming up with each deployment 14:28:47 and integrated to ODL 14:28:49 ? 14:28:49 yes 14:28:59 ODL running on controller1 14:29:05 * ChrisPriceAB It should deploy with ODL and be validated in that state. That's our target 14:29:07 next: does it make sense to test Tempest in that setup? 14:29:12 all OVS switches (5 of them) are connected to ODL 14:29:23 all OVS switches meaning br-int 14:29:25 as far as I understand from Daniel, It doesnt 14:29:38 yes it does 14:29:55 thats the point of Arno is ODL + O/S integration testing 14:29:58 since we are the first to do it 14:30:00 well. on FUEL not yet.. i need to get in (it would seem to login myself) to see how they are doing things to bring it up iun the same fashion 14:30:03 we need to shake those bugs out 14:30:14 * ChrisPriceAB aricg the rebase worked fine. 14:30:18 however, this means that when testing has been running, its been doing the full suite in ODL/VXLAN mode right? 14:30:26 right 14:30:30 thats what I assume yes 14:30:40 so then Jose, our approach has been wrong 14:30:50 and there are bugs in ODL not setting up vxlan tunnels correctly 14:30:53 if the goal is to have semblance between the two 14:30:58 Im just wondering if some of the tempest test cases have failed because of having ODL (hopefully not because of that) 14:31:16 yes Jose - thats why the tempatest have been failing moresore on POD2 than on POD1 14:31:25 cause on POD1, you are doing it in VLAN segregation mode 14:31:29 not with VXLAN tunnel setup at all 14:31:43 and for FUEL, im telling you that when you go full funtest in ODL/VXLAN mode its going to be worse 14:31:44 ok 14:31:50 we are using a really really old OVS 14:31:54 yes, I understood that 14:31:58 so .. we need to reset the dial on all tests 14:31:59 i think 14:32:00 thats why I proposed having 4 steps 14:32:07 cause there is no clarity on the setups you have been testing 14:32:17 you have already been testing FULL TEST with VXLAN/ODL 14:32:17 1) Deploy 2) Run functes SET1 3) Switch to ODL + reboot (if nonHA deployment) 4) Run functest SET2 (if nonHA deployment) 14:32:20 no 14:32:27 what is the point of switching 14:32:28 ? 14:32:39 mmm 14:32:42 CHris has stated that the result is Opesntack with ODL(VXLAN) 14:32:52 * ChrisPriceAB can I have a BGS committer push the amended patch for https://gerrit.opnfv.org/gerrit/#/c/673/. Only change since last commit is the rebase. 14:32:55 +1 theres no point in doing anything non ODL 14:32:57 right 14:33:14 but that is why you said it would take 3 weeks work? 14:33:28 or it is something quicker to fix? 14:33:30 correct.. if you want the full test (which is what i understand you want) 14:33:44 i dont know Jose, how goes the fixing of the bugs on FOREMAN? 14:33:49 jose_lausuch, lmcdasm: Isn't it best to run the tests on the deployed state. Otherwise we don't know the state of the tempest tests after the ODL integration... 14:33:50 Daniel, I dont want anything, I just want to get done what is required by the project 14:33:50 that is the question 14:34:10 ok.. i dont know what to say 14:34:11 we file the bugs with the appropriate software 14:34:15 ive filed a bug on ODL 14:34:20 for O/S integration already 14:34:21 1) Deploy 2) Switch to ODL + reboot (if nonHA deployment) 3) Run functest all (if nonHA deployment) 14:34:22 we need to file more 14:34:33 Tim - go you guys have HA working with ODL at all? 14:34:39 if not, then we can stop talking about HA and non-HA 14:34:49 HA ODL does not work in Helium 14:34:55 O/S HA + non-HA ODL (HA ODL will only be available in Lithium) 14:35:01 but let me clarify something 14:35:06 are we talking now the whole time about Arno? 14:35:09 thanks Chris- however, im talking about at the Ostack level with HAProxy 14:35:11 yes 14:35:22 not about the native ODL support in ODL 14:35:41 Fuel is running non-HA O/S + ODL, when you do that you are going to have a lot more passes than HA O/S+ODL 14:35:46 as far as tempest testing 14:35:54 so its not apples to apples to compare foreman and fuel results 14:35:55 no we arent 14:36:03 we havent run any tempets with ODL turned on 14:36:08 I 14:36:08 ChrisPriceAB: step 3 of your statement is not clear 14:36:09 its only been VLAN Seg deployment 14:36:16 lmcdasm: I'm saying if you do run 14:36:19 if its a HA deployment we dont run functest at all? 14:36:23 with the identical setup to us 14:36:30 except non -HA O/S 14:36:35 you will have more passes 14:36:40 the reason is 14:36:58 because ODL has trouble figuring out where vxlan endpoints should be, when you have that many OVS instances 14:37:18 2 compute ovs instances + 1ovs instance (NON HA) vs 2 compute ovs instances and 3 ovs instances on controllers (HA) 14:37:33 so folks shouldnt compare results from pod 2 to pod 1 14:37:34 * ChrisPriceAB jose_lausuch: we just need to run the same suite against POD1 and POD2. (if we don't have OS HA on POD1 the results may differ but that's OK=) 14:37:47 right 14:37:49 ok 14:37:56 then dig into test cases 14:37:58 figure out why they fail 14:37:58 as long it is documented, right? 14:37:59 file bugs 14:38:04 yeah 14:38:07 we need to know why they fail 14:38:15 it cant be because an installer installed wrong 14:38:20 it needs to be a real bug 14:38:25 then I would correct it to: 1) Deploy 2) Switch to ODL + reboot (if nonHA deployment) 3) Run functest all 14:38:35 nope Jose 14:38:40 you need two different approaches 14:38:45 for FOREMAN ODL is deployed 14:38:49 for FUEL you will need to switch 14:39:00 yes, step2 14:39:05 * ChrisPriceAB yes but before the tests are run 14:39:18 are we all agreed that we will run no TESTING again anything without ODL / VXLAN turned on? 14:39:20 * ChrisPriceAB that's part of the deploy flow 14:39:35 no Jose - FOREMAN deploys ODL nativelly. so you do 1) Deploy and 3) Run test all 14:39:37 no switching 14:39:46 Im talking about fuel only 14:39:52 I know there is no switch in Foreman 14:39:56 sorry for not being clear 14:40:12 FUEL --> 1) Deploy 2) Switch to ODL + reboot (if nonHA deployment) 3) Run functest all 14:40:25 FOREMAN --> 1) Deploy 2) Run functest all 14:40:25 well.. now it begs a real questino 14:40:29 exactly 14:40:34 that comes your concern 14:40:39 then why not change FUEL puppet to simply deploy in ODL mode then 14:40:52 cause seems to me that we have now done weeks of testing on an object that we dont want 14:41:03 having Fuel with nonHA+ODL, you said Temepst will not work 14:41:08 nope 14:41:09 so what is the poing of that? 14:41:19 point* 14:41:20 it needs work - wont work as it is today 14:41:25 excactly 14:41:36 so ChrisPriceAB how do we handle that? 14:41:38 so.. if you want to run the switch 14:41:40 what is the focus for Arno? 14:41:40 then thats fine. 14:41:59 * ChrisPriceAB can you state the question concisely? 14:42:00 if the goal is testing results for Ostack with ODL then you need to reset the counters on all your FUEL testing 14:42:22 why does ChrisPriceAB always use /me ? :) 14:42:33 * ChrisPriceAB is very shy 14:42:41 haha 14:42:49 but /me is for 3rd person context 14:42:52 like this 14:42:54 * trozet runs to the store 14:43:14 yeah looks like I ask myself questions... 14:43:21 haha 14:43:45 so, Jose i think CHris is waiting for you to re-ask the question 14:44:00 jose_lausuch: focuse for Arno is OPNFV which at the moment is ODL + O/S, and eventuall service function changing (SFC) enabled in ODL 14:44:01 Solution A) 1) Deploy 2) Functest without ODL 3) Switch to ODL 4) test ODL 14:44:01 Solution B) 1 ) Deploy 2) Switch to ODL) Test full Functest 14:44:12 Solution B needs more work and cannot make it for Arno 14:44:25 * caveat - 14:44:50 <[1]JonasB> So realistically I think we have a situation for Fuel that we either run tempest with non ODL configuration + a limited ODL test and fix the rest for SR1, or we do not release fuel the 4'th. 14:45:04 Can I ask why we can't run the tests the same way on both PODs? 14:45:19 Daniel, shoot 14:45:24 my final comment there is no defined exit criteria for relase - with this approach, i can say sure its done and then mark things as "bugs" 14:45:41 when / how are you drawing the line on what is acceptable for release and not? 14:45:56 me? 14:46:01 Answer Chris - see above Tim explained it 14:46:18 as well that is what i have been trying to get infomraino for so I can make it the same and we can compare apples to apples 14:46:32 but seems this is not going to be possible. 14:47:27 Ok, so tim explained that running OS non-HA will result in better results on POD1. Is that a blocking issue? 14:48:12 So we dont focus on testing Fuel HA? 14:48:24 well there was a bit more than that.. bsaically about the vxlan tunnel 14:48:37 * ChrisPriceAB is very confused because he understood we could not get Fuel HA working with ODL. 14:48:46 anyway - its moot.. the point being how you want to proceed at this point. we have a month of testing on FUEL where we havent been testing the right objec 14:48:54 thats not true Chris - 14:49:16 so.. we can try to set the FUEL to the same configuration as FOREMAN 14:49:25 (once the POD2 is up and i can go pick the pieces that we are looking for) 14:49:27 * ChrisPriceAB by next week, but that we could get Fuel single instance & ODL working by next week. 14:49:35 well - please define working? 14:49:40 since we had it working for 4 months 14:49:54 <[1]JonasB> lmcdasm: we havent had month of testing - comfused :-/ 14:49:55 where the scope wsa simply to create netowrking and see them reflected in ODL gui 14:50:00 now the scope had changed 14:50:11 btw, deployment is completed 14:50:49 lets talk after the meeting Daniel to get the needed information 14:50:52 <[1]JonasB> I need to run for a meeting 14:51:05 <[1]JonasB> #endmeeting