14:05:04 <mbeierl> #startmeeting Test and Performance Weekly Meeting
14:05:04 <collabot`> Meeting started Thu Jun 16 14:05:04 2016 UTC.  The chair is mbeierl. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:05:04 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:05:04 <collabot`> The meeting name has been set to 'test_and_performance_weekly_meeting'
14:05:27 <mbeierl> #topic Roll Call
14:05:29 <morgan_orange> #topic call role
14:05:29 <kubi001> #info kubi
14:05:31 <mbeierl> #info Mark Beierl (StorPerf)
14:05:38 <yujunz> #info Yujun Zhang
14:05:39 <morgan_orange> #info Morgan Richomme (Functest)
14:05:39 <mbeierl> #chair morgan_orange
14:05:39 <collabot`> Current chairs: mbeierl morgan_orange
14:05:49 <mbeierl> any other takers for chair?
14:06:51 <Julien-zte> #info Julien
14:07:24 <mbeierl> #topic QTIP Presentation
14:07:57 <mbeierl> I am doing #topic incorrectly too?
14:08:05 <mbeierl> #help
14:08:08 <mtahhan> #info Maryam Tahhan
14:08:15 <mtahhan> on IRC only today
14:08:30 <mtahhan> mbeierl: I think you have to be chair
14:08:31 <yujunz> #link https://wiki.opnfv.org/display/qtip
14:08:35 <mbeierl> #info QTIP being discussed
14:09:11 <mbeierl> mtahhan: I am chair, but it does not seem to be responding to help or topic...  Maybe collabot is a little sleepy
14:09:13 <dneary> mbeierl, You're doing it right, I think
14:09:51 <mbeierl> mtahhan: the QTIP link ^ is what is being displayed on the GTM.
14:10:15 <mbeierl> #info discussion of QTIP scope
14:11:03 <yujunz> I lost the audio
14:11:14 <mbeierl> #info open question after last demo: is QTIP going to continue independently or should it roll up into Functest and Yardstick
14:11:17 <mtahhan> I think there was a similar issue with meetbot just last week
14:11:24 <mtahhan> dun dun duuuuuuun
14:11:36 <mbeierl> mtahhan: I think so...
14:11:37 <Julien-zte> #info morgan_orange mentioned what's the overlap between qtip and other projects such as bottleneck
14:12:09 <mbeierl> #info yujunz discussed QTIP internally and is proposing a roadmap.
14:12:24 <morgan_orange> Julien-zte: my concern is more with yardstick, bottlenecks is a bit different
14:13:02 <yujunz> #link https://gerrit.opnfv.org/gerrit/#/c/15681/4/docs/roadmap.rst,unified
14:13:03 <Julien-zte> morgan_orange, qtip care about the performance
14:13:06 <mbeierl> #info new PTL, new members, so new roadmap is proposed, please comment on the roadmap
14:13:50 <mbeierl> #info morgan_orange asked if QTIP tests *before* the OPNFV installation?
14:13:57 <Julien-zte> from the begin, yardstick and functest is more about the feature test
14:14:03 <kubi001> Julien-zte: yardsitck also care about performance
14:14:21 <Julien-zte> I don't know the scope changes
14:14:50 <kubi001> Julien-zte: performance testing is the key testing area since yardstick setup
14:14:57 <mbeierl> #info yardstick has now confirmed as being umbrella for all performance testing - the ones built in, and well as the interface to CPerf, StorPerf, and VSPerf
14:15:10 <morgan_orange> Julien-zte: Functest = functional testing, Yardstick = ërformance
14:15:21 <mbeierl> #info Functest is umbrella for all functional tests
14:15:47 <yujunz> I can't hear anythiing
14:16:00 <Julien-zte> from the begin, functest test on the infrastructure, yardstick perform the test as a tenant in a VM
14:16:28 <Julien-zte> it is the place we can discuss more clear about the overlap
14:16:48 <Julien-zte> let's avoid overlapping work in the future
14:18:02 <mbeierl> #info During Colorado planning, there has been a discussion of test projects to attempt to clear the overlap.  It comes down to all projects should be affiliated with one of the two umbrellas: performance (Yardstick) or functional (Functest)
14:18:57 <Julien-zte> who can provide the info about yardstick become the umbrella for other test projects
14:19:04 <mbeierl> #info that does not mean that individual projects go away.  VSPerf is very much independent, but it's OPNFV reporting and CI pipeline are going to be driven from Yardstick integration
14:19:32 <mbeierl> mtahhan: can clarify that statement if needed.  I just don't always want to talk about myself
14:19:34 <mtahhan> hopefully :P
14:19:43 <mbeierl> ditto for StorPerf
14:19:55 <kubi001> Julien-zte:  let talk in berlin
14:20:00 <Julien-zte> good
14:20:02 <yujunz> Sure
14:20:39 <mbeierl> #info agreed that this conversation will continue in Berlin: "How does QTIP fit in now that there is the plan to have the two major umbrella projects"
14:21:20 <Julien-zte> mbeierl, morgan_orange, let's have a discuss on the summit
14:21:29 <Julien-zte> and kubi001
14:21:59 <mbeierl> #link https://wiki.opnfv.org/display/functest/testing
14:22:21 <mbeierl> Discussing the overview picture
14:23:31 <mbeierl> #link https://wiki.opnfv.org/download/attachments/2926690/Testing.png?version=1&modificationDate=1463478046000&api=v2
14:24:20 <mbeierl> #link https://wiki.opnfv.org/download/attachments/2926690/how_to_testing.png?version=1&modificationDate=1464962177000&api=v2 "How to interact with other projects"
14:24:32 <mbeierl> #topic "Conversations with the test teams"
14:25:01 <mbeierl> #info to introduce ourselves to attendees - perhaps upstream contributors
14:25:24 <mbeierl> #info to discuss our goals and ambitions (collective and individual) for the testing projects
14:25:48 <frankbrockners> Hey folks... I would suggest that we re-draw this picture to make it comprehensive - i.e. include the earlier presented Qtip.
14:26:03 <frankbrockners> In addition, does Bottlenecks really feed into Yardstick?
14:26:31 <mbeierl> frankbrockners: During the plugfest, it appeared that Bottlenecks can use information from Yardstick
14:26:45 <mbeierl> so it draws info, not publishes info
14:31:16 <mbeierl> #info discussion about the https://wiki.opnfv.org/download/attachments/2926690/Testing.png?version=1&modificationDate=1463478046000&api=v2 picture
14:32:02 <mbeierl> #info it's a way of looking at the testing framework
14:32:24 <mbeierl> #info Presentation is in git, team leads to add content
14:32:50 <mbeierl> #info Goal is to emphasize Functest is upstream focused
14:33:45 <mbeierl> #info, and that it's time to give back :)
14:34:04 <mbeierl> #info Open the discussion about defining Telco KPIs
14:35:03 <frankbrockners> mbeierl - thanks - so sounds like the arrow points the wrong direction
14:36:07 <frankbrockners> mbeierl - I also thought that vsperf publishes directly into MongoDB - and does not go via Yardstick to do so - but mtahhan might correct me here
14:36:35 <mbeierl> frankbrockners: I think so.  Either that or Bottlenecks indicated it's intention to read from different sources and use Yardstick for reporting to the common DB.
14:37:19 <mbeierl> frankbrockners: you'll notice that VS, C, and Stor-Perf all have double arrows.  Yardstick can kick them off, and they can report back up
14:37:35 <Julien-zte> mbeierl, in the summit we can discuss about the where to store and how to demonstrate the result
14:38:41 <mbeierl> #info Daniel clarified that the diagrams do not constitute formal organization, but rather a mutual agreement on presentation of how things fit together
14:39:53 <frankbrockners> mbeierl - thanks - also just checked with mtahhan that vsperf will continue to publish directly into mongodb - so the real picture would look a bit more "complex" :-)
14:40:39 <morgan_orange> frankbrockners: yardstick can publish to mongodb/influxDB
14:41:07 <morgan_orange> but that is why we put several DBs in the figure
14:41:36 <frankbrockners> yup -- but you probably need more arrows in your picture ;-)
14:41:37 <mbeierl> right, but to frankbrockners point, there are no "direct lines" from the *Perf projects to the DB
14:41:49 <morgan_orange> ah ok right
14:41:54 <kubi001> frankbrockners:  vsperf can  publish directly into mongodb, it also can go via Yardstick when Vsperf integrate with Yardstick. It depends on Vsperf
14:42:41 <kubi001> same with Storperf
14:42:45 <mbeierl> kubi001: that is also a question that StorPerf has - we use a private Carbon (Graphite) DB
14:42:56 <mbeierl> kubi001: exactly, thanks!
14:44:37 <kubi001> mbeierl: Carbon DB + Grafana in StorPerf?
14:45:15 <mbeierl> kubi001: correct: https://wiki.opnfv.org/display/storperf/Storperf+Architecture
14:46:32 <kubi001> mbeierl: great
14:46:41 <mbeierl> kubi001: Right now, I summarize the findings into an average at the end of the run, and only publish that summary result to the MongoDB via the test API
14:47:42 <mbeierl> kubi001: but I need to decide where the summary should be done - in StorPerf, or in the Dashboards later?  How much data do I want people to store in to common DB?  This can be gigabytes worth of data per run...
14:48:45 <mbeierl> My thought is: StorPerf exports the Graphite (and I one day will add Grafana) via a web UI.  The raw data can be examined in the container.  However, the final test result will be the summary only to save space.
14:50:36 <kubi001> mbeierl: I see. Same situation with yardstick, so we support MongoDB via test API, and also support InfluxDB + Grafana to support more detail result
14:51:39 <mbeierl> dfarrell07, mtahhan: what are you planning for your raw data vs. summary results for longer term storage and publication?
14:54:01 <dfarrell07> mbeierl: we haven't gotten that far, but we don't expect to have tons of data for the tools we use so far, so we will not have that problem
14:54:12 <mbeierl> nice :)
14:55:00 <kubi001> :)
14:55:41 <mbeierl> #action: morgan_orange to create a page for Berlin summit test group planning
14:59:15 <mbeierl> #info Will discuss during the summit how to have rolling organization of this weekly meeting so that it becomes more collaborative
14:59:48 <mbeierl> #info Project leads, etc, encouraged to fill in details in the presentation
15:00:02 <kubi001> bye
15:00:10 <mbeierl> #endmeeting