#opnfv-cperf: cperf

Meeting summary

1. 1. jamoluhrsen got upstream tweaks to run robot scale tests in cperf pod (mattw4, 18:18:48)
   2. sai started latest round of scale testing (mattw4, 18:20:08)
   3. env is running with clustered ODL (mattw4, 18:20:31)
   4. started with 3 controllers & 3 computes (mattw4, 18:21:11)
   5. https://nexus.opendaylight.org/content/repositories/opendaylight-carbon-epel-7-x86_64-devel/org/opendaylight/integration-packaging/opendaylight/6.2.0-0.1.20170824rel1941.el7.noarch/opendaylight-6.2.0-0.1.20170824rel1941.el7.noarch.rpm Sai may want to consider using this RPM, it has no-IP fix Tim wanted for OPNFV (dfarrell07, 18:21:18)
   6. much harder to hit OOM this time (with ODL defaults) (mattw4, 18:21:42)
   7. encountering bugs similar to memory leaks & Java failures seen previously (mattw4, 18:22:45)
   8. https://bugs.opendaylight.org/show_bug.cgi?id=9034 (sai, 18:22:51)
   9. ODL version == prerelease SR2 build (mattw4, 18:23:02)
   10. ODL should have memory leak fix, but still may fail clustering (mattw4, 18:23:42)
   11. we likely have more memory leaks to fix (mattw4, 18:24:39)
   12. second issue under investigation is networking-odl driver itself (mattw4, 18:25:43)
   13. journal driver getting hit hard (mattw4, 18:25:55)
   14. workaround is to reduce number of worker processes (mattw4, 18:26:23)
   15. mysql has several queries that are taking a long time and consuming a lot of resources (mattw4, 18:27:04)
   16. browbeat tool helping to monitor resources, but missing java info - sai has added a plugin (mattw4, 18:28:03)
   17. need deep-dive into java resource consumption to understand better (mattw4, 18:28:46)
   18. https://snapshot.raintank.io/dashboard/snapshot/YGlm4u58NFJzMzgNhLYDiYGHX2pFKUdy?orgId=2&panelId=226&fullscreen&from=1503579973482&to=1503583573482&var-Cloud=odl-osp12-non-containerized&var-NodeType=*&var-Node=overcloud-controller-2&var-Interface=interface-br-ex&var-Disk=disk-sda&var-OSD=All&var-Pool=All&var-cpus0=All&var-cpus00=All (sai, 18:29:13)
   19. https://snapshot.raintank.io/dashboard/snapshot/emsmNggId7uHKq9S1r2xX5qhBew2GCP6?orgId=2&panelId=69&fullscreen&from=1503595117077&to=1503598717077&var-Cloud=odl-osp12-non-containerized&var-NodeType=*&var-Node=overcloud-controller-0&var-Interface=interface-br-ex&var-Disk=disk-sda&var-OSD=All&var-Pool=All&var-cpus0=All&var-cpus00=All (sai, 18:29:33)
   20. sai shared lovely grafana dashboards showing resource consumption (mattw4, 18:30:41)
   21. sai still seeing lots of slow queries to mysql (mattw4, 18:33:47)
   22. mattw4 to take AR to follow up with LF (mattw4, 18:34:25)
   23. mattw4 needs to learn more about robot (mattw4, 18:35:24)
   24. mattw4 to ping sai for help on monitoring (mattw4, 18:37:38)

Action items

1. (none)

People present (lines said)

1. mattw4 (22)
2. sai (4)
3. collabot (3)
4. dfarrell07 (1)