Thanks Ali.


@Ashutosh Gupta<mailto:<email address hidden>> please work with Vivek to triage the issue on scale out.

________________________________
From: Ali Zaringhalam
Sent: Wednesday, September 26, 2018 7:56:01 AM
To: Dilip Sundarraj; Shivayogi Ugaji; Ashutosh Gupta
Cc: Radosław Romaszewski; <email address hidden>; Sarin Kizhakkepurayil; Linh Phan; Sajeesh Mathew
Subject: RE: 5.0.01 /Try Upgrade and Documentation - Update on Contrail Command Issues


+ Linh, Sajeesh



Hi Dilip:



We will set up a POD and let you know.



I would also like to bring to your attention a bug we found in running our auto-scaling demo.  Briefly, after scale-out traffic does not flow to the new vSRX.  The workaround is:



·         Delete port-tuple and add a new one. (This points to a bug.)

·         We also use HealthCheck to ensure traffic flows when the vSRX interface is reached after boot.



Sajeesh is investigating this in our setup and will create a ticket in Launchpad when done.  We went back to 3.2.3 and found the same issue, so the bug is currently in the field.  This issue should be fixed in 5.0.2.   It will likely require a back-end change in Contrail.



Regards,



Ali



From: Dilip Sundarraj
Sent: Wednesday, September 26, 2018 12:44 AM
To: Shivayogi Ugaji <email address hidden>; Ashutosh Gupta <email address hidden>; Ali Zaringhalam <email address hidden>
Cc: Radosław Romaszewski <email address hidden>; <email address hidden>; Sarin Kizhakkepurayil <email address hidden>
Subject: Re: 5.0.01 /Try Upgrade and Documentation - Update on Contrail Command Issues



Hi Ali



Could you share the access to the setup with this issue https://bugs.launchpad.net/juniperopenstack/+bug/1792650 ?



-Dilip



________________________________

From: Shivayogi Ugaji
Sent: Tuesday, September 25, 2018 9:23 PM
To: Ashutosh Gupta
Cc: Radosław Romaszewski; <email address hidden><mailto:<email address hidden>>; Dilip Sundarraj; Sarin Kizhakkepurayil
Subject: Re: 5.0.01 /Try Upgrade and Documentation - Update on Contrail Command Issues



Hi Ashutosh,



Do you have a setup for a quick look ?



Thanks

Shivayogi



From: Ashutosh Gupta <<email address hidden><mailto:<email address hidden>>>
Date: Tuesday, September 25, 2018 at 1:25 PM
To: Shivayogi Ugaji <<email address hidden><mailto:<email address hidden>>>
Cc: Radosław Romaszewski <<email address hidden><mailto:<email address hidden>>>, "<email address hidden><mailto:<email address hidden>>" <<email address hidden><mailto:<email address hidden>>>, Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>, Sarin Kizhakkepurayil <<email address hidden><mailto:<email address hidden>>>
Subject: Re: 5.0.01 /Try Upgrade and Documentation - Update on Contrail Command Issues



Hi Shivayogi



Any update on the bug?



Thanks,

Ashutosh





From: Shivayogi Ugaji <<email address hidden><mailto:<email address hidden>>>
Date: Monday, September 24, 2018 at 10:41 PM
To: Sarin Kizhakkepurayil <<email address hidden><mailto:<email address hidden>>>, Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>
Cc: Ashutosh Gupta <<email address hidden><mailto:<email address hidden>>>, Radosław Romaszewski <<email address hidden><mailto:<email address hidden>>>, "<email address hidden><mailto:<email address hidden>>" <<email address hidden><mailto:<email address hidden>>>
Subject: Re: 5.0.01 /Try Upgrade and Documentation - Update on Contrail Command Issues



Sarin,



I assumed to be old UI issue based on attached Ashutosh response, will take a look tomorrow.



Regards

Shivayogi



From: Sarin Kizhakkepurayil <<email address hidden><mailto:<email address hidden>>>
Date: Monday, September 24, 2018 at 10:09 PM
To: Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>
Cc: Shivayogi Ugaji <<email address hidden><mailto:<email address hidden>>>, Ashutosh Gupta <<email address hidden><mailto:<email address hidden>>>, Radosław Romaszewski <<email address hidden><mailto:<email address hidden>>>, "<email address hidden><mailto:<email address hidden>>" <<email address hidden><mailto:<email address hidden>>>
Subject: Re: 5.0.01 /Try Upgrade and Documentation - Update on Contrail Command Issues



Guys this is a config issue or something in service instance launching side which creates Vnc objects. Please go through the bug details. Sorry I didn’t check if it got reassigned back to me.

There’s no page in old UI which could’ve exposed this as we never displayed workloads and if you examine the instance-ip object, it has some garbage value.



Why do you say it’s UI bug?



Sent from my iPhone

On Sep 24, 2018, at 9:51 PM, Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>> wrote:

Thanks Yogi.



@Sarin Kizhakkepurayil<mailto:<email address hidden>> please let us know, we’d like to deploy this in the /try labs as soon as we can.



From: Shivayogi Ugaji <<email address hidden><mailto:<email address hidden>>>
Date: Monday, September 24, 2018 at 9:48 PM
To: Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>, Sarin Kizhakkepurayil <<email address hidden><mailto:<email address hidden>>>, Ashutosh Gupta <<email address hidden><mailto:<email address hidden>>>, Radosław Romaszewski <<email address hidden><mailto:<email address hidden>>>, "<email address hidden><mailto:<email address hidden>>" <<email address hidden><mailto:<email address hidden>>>
Subject: Re: 5.0.01 /Try Upgrade and Documentation - Update on Contrail Command Issues



Dilip,



It was re-assigned to Sarin last week as the issue was seen only with new UI and not with the old UI.



Shivayogi



From: Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>
Date: Monday, September 24, 2018 at 8:17 PM
To: Sarin Kizhakkepurayil <<email address hidden><mailto:<email address hidden>>>, Ashutosh Gupta <<email address hidden><mailto:<email address hidden>>>, Radosław Romaszewski <<email address hidden><mailto:<email address hidden>>>, "<email address hidden><mailto:<email address hidden>>" <<email address hidden><mailto:<email address hidden>>>, Shivayogi Ugaji <<email address hidden><mailto:<email address hidden>>>
Subject: Re: 5.0.01 /Try Upgrade and Documentation - Update on Contrail Command Issues



@Sarin Kizhakkepurayil<mailto:<email address hidden>> @Shivayogi Ugaji<mailto:<email address hidden>> Any updates on these issues?



-Dilip

________________________________

From: Dilip Sundarraj
Sent: Thursday, September 20, 2018 3:34:35 PM
To: Sarin Kizhakkepurayil; Ashutosh Gupta; Radosław Romaszewski; <email address hidden><mailto:<email address hidden>>; Shivayogi Ugaji
Subject: Re: 5.0.01 /Try Upgrade and Documentation - Update on Contrail Command Issues



Thank you Sarin. Will wait for your fix for issue 1.



@Shivayogi Ugaji<mailto:<email address hidden>> would you please provide us a fix asap, this is blocking us from deploying 5.0.1 in our sandbox labs. We will plan to upgrade to 5.0.2 when that is released.



From: Sarin Kizhakkepurayil <<email address hidden><mailto:<email address hidden>>>
Date: Thursday, September 20, 2018 at 11:29 AM
To: Ashutosh Gupta <<email address hidden><mailto:<email address hidden>>>, Radosław Romaszewski <<email address hidden><mailto:<email address hidden>>>, "<email address hidden><mailto:<email address hidden>>" <<email address hidden><mailto:<email address hidden>>>
Cc: Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>
Subject: Re: 5.0.01 /Try Upgrade and Documentation - Update on Contrail Command Issues





For the instance status we’ve a PR raised to fix the issue. It needs to handle 1 more additional case.

https://github.com/Juniper/contrail-ui/pull/1449. ETA mostly tomorrow.



for the second IP display issue, UI seems to be doing the right thing. Config object has the IP address in weird value. I’ve moved to Shivayogi (config team)

I’ve updated bug with details.



From: Ashutosh Gupta <<email address hidden><mailto:<email address hidden>>>
Date: Thursday, September 20, 2018 at 7:43 AM
To: Sarin Kizhakkepurayil <<email address hidden><mailto:<email address hidden>>>, Radosław Romaszewski <<email address hidden><mailto:<email address hidden>>>
Cc: Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>
Subject: Re: 5.0.01 /Try Upgrade and Documentation - Update on Contrail Command Issues



Hey Sarin



Could you please provide me the status of the bugs we filed? (Contrail Command UI)



  1.  https://bugs.launchpad.net/juniperopenstack/+bug/1791842



  1.  https://bugs.launchpad.net/juniperopenstack/+bug/1792650



Thanks,

Ashutosh





From: Sarin Kizhakkepurayil <<email address hidden><mailto:<email address hidden>>>
Date: Friday, September 14, 2018 at 11:00 AM
To: Ashutosh Gupta <<email address hidden><mailto:<email address hidden>>>, Radosław Romaszewski <<email address hidden><mailto:<email address hidden>>>
Cc: Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>
Subject: Re: 5.0.01 /Try Upgrade and Documentation - Update on Contrail Command Issues



+Radek, please take a look at the issue below



Ashutosh, could you open up a bug and assign to Radek. He will get it fixed. Thanks



From: Ashutosh Gupta <<email address hidden><mailto:<email address hidden>>>
Date: Friday, September 14, 2018 at 10:01 AM
To: Sarin Kizhakkepurayil <<email address hidden><mailto:<email address hidden>>>
Cc: Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>
Subject: FW: 5.0.01 /Try Upgrade and Documentation - Update on Contrail Command Issues



Hello Sarin



Can you please take a look at the issue in the trailing mail.



Thanks,

Ashutosh





From: Linh Phan <<email address hidden><mailto:<email address hidden>>>
Date: Thursday, September 13, 2018 at 12:02 PM
To: Ali Zaringhalam <<email address hidden><mailto:<email address hidden>>>, Ashutosh Gupta <<email address hidden><mailto:<email address hidden>>>, Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>
Cc: Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>
Subject: RE: 5.0.01 /Try Upgrade and Documentation - Update on Contrail Command Issues



Hi Ashutosh,



I wonder if this is an expected behavior of CC



For the vSRX firewalls as “Service_chaining”, under tab Instances, it shows the IP address in this strange format when I hover my mouse to both firewall instance ? [id:image001.png@01D45457.A88678E0]

[id:image002.png@01D45457.A88678E0]





And the only place under Instances tab that I can look for the correct IP address (to SSH to it) is under the json snippet



[id:image003.png@01D45457.A88678E0]





Meanwhile in Horizon it shows the IPs as normal



[id:image004.png@01D45457.A88678E0]

[id:image005.png@01D45457.A88678E0]



Only the GUI display have problem. All instances are running and networking correctly.



Thanks.



Regards,

Linh.



From: Ali Zaringhalam
Sent: Wednesday, September 12, 2018 7:10 PM
To: Ashutosh Gupta <<email address hidden><mailto:<email address hidden>>>; Linh Phan <<email address hidden><mailto:<email address hidden>>>; Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>
Cc: Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>
Subject: RE: 5.0.01 /Try Upgrade and Documentation - Update on Contrail Command Issues



Thanks Ashutosh.  I am following progress on related threads.  Also, as discussed today, we look forward to information on how to pass Cassandra parameters (e,g., heap size) to the Cassandra containers before they are started.



Regards,



Ali



From: Ashutosh Gupta
Sent: Wednesday, September 12, 2018 8:00 PM
To: Ali Zaringhalam <<email address hidden><mailto:<email address hidden>>>; Linh Phan <<email address hidden><mailto:<email address hidden>>>; Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>
Cc: Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>
Subject: FW: 5.0.01 /Try Upgrade and Documentation - Update on Contrail Command Issues



Hello Ali/Chaitanya/Linh



I just had a talk with Sarin and this is what he said for the following issues

  1.  Contrail Service Chain inactive alarm on Contrail Command GUI.

Bug Filed

https://bugs.launchpad.net/juniperopenstack/+bug/1791842

(Might need a week or so to resolve)



  1.  This VNC params in nova.conf is not taking effects. This makes it not possible to directly access instance’s VNC console, from both Horizon/ContrailCommand.

Ignitious is having a look at this issue. I should be able to update you on this by tomorrow EOD.







From: Ashutosh Gupta <<email address hidden><mailto:<email address hidden>>>
Date: Monday, September 10, 2018 at 10:32 AM
To: Ali Zaringhalam <<email address hidden><mailto:<email address hidden>>>, Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>, Linh Phan <<email address hidden><mailto:<email address hidden>>>
Cc: Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>
Subject: Re: 5.0.01 /Try Upgrade and Documentation - Working Session



Hello Ali/Chaitanya/Linh



As per our conclusions from the meeting, these the issues we are left to address.

  1.  Contrail Service Chain inactive alarm on Contrail Command GUI.
  2.  This VNC params in nova.conf is not taking effects. This makes it not possible to directly access instance’s VNC console, from both Horizon/ContrailCommand.



As discussed, I will provide the setup access to Sarin and Ignatious and they will look into the issue. I will follow up with him thereafter.



Thanks,

Ashutosh



From: Ali Zaringhalam <<email address hidden><mailto:<email address hidden>>>
Date: Monday, September 10, 2018 at 8:59 AM
To: Ashutosh Gupta <<email address hidden><mailto:<email address hidden>>>, Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>, Linh Phan <<email address hidden><mailto:<email address hidden>>>
Cc: Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>
Subject: RE: 5.0.01 /Try Upgrade and Documentation - Working Session



Hi Ashutosh:



Please add the Service Chain issue Linh (see his email) identified for Contrail Command.



Regards,



Ali



From: Ashutosh Gupta
Sent: Monday, September 10, 2018 11:35 AM
To: Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>; Ali Zaringhalam <<email address hidden><mailto:<email address hidden>>>; Linh Phan <<email address hidden><mailto:<email address hidden>>>
Cc: Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>
Subject: Re: 5.0.01 /Try Upgrade and Documentation - Working Session



Hello Chaitanya/Ali/Linh



Sorry for the delayed response.



List of Issues

  1.  How do I change/increase the timeout for Horizon/Contrail GUI ? Many users have complained about the GUI keep timing out while they’re doing demo.

               UI works based on keystone token expiry. Extending default expiry on keystone should work (UI team has not tested this)



  1.  Is there an easy way to add customized “nova.conf” params when installing with Contrail Command ?

For example, if manually install with ansible-deployer, the instances.yaml have this section. The only way I can do it now is by manually add those line in all nova.conf

kolla_config:

  openstack_release: ocata

  customize:

    nova.conf: |

      [oslo_messaging_notifications]

      driver = messagingv2



      [DEFAULT]

      disk_allocation_ratio = 20.0

      ram_allocation_ratio = 5.0

      cpu_allocation_ratio = 25.0



              From UI setup wizard, under advanced options in orchestrator step, you could set kolla globals key value params. but I believe sending second level key value will not          work (Ram/Ignatious still need to confirm this)



  1.  This VNC params in nova.conf is not taking effects. This makes it not possible to directly access instance’s VNC console, from both Horizon/ContrailCommand.

Under Investigation from the engineering team.



  1.  Memory: with 64 G RAM per cfg/ctlr/analytics node, we expect the number of PODs per server to be reduced to 2-3 from the current 5 PODs/server in Contrail 3.2.3.  This is a significant reduction and will impact the intended applications (external users and training).  We have asked engineering to investigate tuning parameters to reduce the RAM utilization.

The difference is micro-services. 32 GB is just about enough to bring up the container. Whether we can fine tune, I still need a confirmation on this from the development team.



  1.  Contrail Command: when we install Contrail Command, we configure the cfg node for CSN.  This causes vRouter to be installed on the cfg node.  We currently don’t plan to use CSN but would like to make sure that the presence of vrouter on cfg node is benign.  Should we remove CSN from the installation altogether?

vRouter is usually not present on the config node. CSN is not necessary for bringing up the cluster and you can remove CSN from the installation.



  1.  “Contrail Command resources: what is the minimum requirement for a VM hosting Contrail Command.  In our setup, each POD is managed by a separate Contrail Command.  We have installed this on a VM with 2 CPU and 2 G RAM and it seems to working fine.  We would like to have confirmation that this is OK.“

2G of RAM is not tested and could be too less. The min recommended is 8GB and 2 CPU.

Linh Phan is testing with 4BG to see if it is required to be upgraded to 8GB.





I will confirm the config node RAM issue with the engineering team today.



Thanks,

Ashutosh









From: Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>> on behalf of Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>
Date: Saturday, September 8, 2018 at 7:48 AM
To: Ashutosh Gupta <<email address hidden><mailto:<email address hidden>>>
Subject: FW: 5.0.01 /Try Upgrade and Documentation - Working Session



Hi Ashutosh



Please provide an update of where things stand and what the open issues are.



-Dilip



From: <email address hidden><mailto:<email address hidden>>
When: 10:00 AM - 11:00 AM September 10, 2018
Subject: 5.0.01 /Try Upgrade and Documentation - Working Session
Location: ~CR-US-SVL-A.6.103-Branch Rickey; Skype





Hi Ashutosh,

+Dilip,



As we discussed on the call, can you talk to Sarin and other folks from engineering who can help with the issues related to Contrail Command and CSN. I will setup a call for Monday between all the people listed on this thread to go over the details.



Thanks
Chaitanya



.........................................................................................................................................

Join Skype Meeting<https://meet.juniper.net/ckadiyala/WZ2Q03HH>

Trouble Joining? Try Skype Web App<https://meet.juniper.net/ckadiyala/WZ2Q03HH?sl=1>

Join by phone



+14089369000<tel:+14089369000> (USA, West Coast)                   English (United States)

+19785898300<tel:+19785898300> (USA, West Coast)                   English (United States)

ILYNC (45962)<tel:ILYNC%20(45962)> (USA, West Coast)                     English (United States)

+18446454398<tel:+18446454398> (USA, West Coast)                   English (United States)

+18332135669<tel:+18332135669> (USA, West Coast)                   English (United States)



Find a local number<https://dialin.juniper.net>



Conference ID: 66533016

 Forgot your dial-in PIN?<https://dialin.juniper.net> |Help<https://o15.officeredir.microsoft.com/r/rlidLync15?clid=1033&p1=5&p2=2009>





Please consider with whom you are communicating, including non-Juniper personnel of companies federated on Skype for Business, before sharing any confidential information.  Non-disclosure agreements may apply.

[!OC([1033])!]

.........................................................................................................................................







_____________________________________________
From: Ali Zaringhalam
Sent: Friday, September 7, 2018 10:34 AM
To: Ashutosh Gupta <<email address hidden><mailto:<email address hidden>>>; Linh Phan <<email address hidden><mailto:<email address hidden>>>
Cc: Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>; Sarin Kizhakkepurayil <<email address hidden><mailto:<email address hidden>>>; Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>; Ignatious Johnson <<email address hidden><mailto:<email address hidden>>>; Ramprakash Ram Mohan <<email address hidden><mailto:<email address hidden>>>; Jerrod Spooner <<email address hidden><mailto:<email address hidden>>>
Subject: RE: 5.0.01 /Try Upgrade and Documentation - Working Session



Hi Folks:



Below are the meeting minutes from the conference call today.



  *   Memory: with 64 G RAM per cfg/ctlr/analytics node, we expect the number of PODs per server to be reduced to 2-3 from the current 5 PODs/server in Contrail 3.2.3.  This is a significant reduction and will impact the intended applications (external users and training).  We have asked engineering to investigate tuning parameters to reduce the RAM utilization.  I am attaching the recent email thread on this topic.
  *   Contrail Command: when we install Contrail Command, we configure the cfg node for CSN.  This causes vRouter to be installed on the cfg node.  We currently don’t plan to use CSN but would like to make sure that the presence of vrouter on cfg node is benign.  Should we remove CSN from the installation altogether?
  *   Console access: the following thread discussed the issue.  We are awaiting a resolution from engineering.
  *   Contrail Command resources: what is the minimum requirement for a VM hosting Contrail Command.  In our setup, each POD is managed by a separate Contrail Command.  We have installed this on a VM with 2 CPU and 2 G RAM and it seems to working fine.  We would like to have confirmation that this is OK.
  *   Chaitanya will set up a follow-up call early next week.



Regards,



Ali



From: Ashutosh Gupta
Sent: Friday, September 7, 2018 12:34 PM
To: Linh Phan <<email address hidden><mailto:<email address hidden>>>
Cc: Ali Zaringhalam <<email address hidden><mailto:<email address hidden>>>; Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>; Sarin Kizhakkepurayil <<email address hidden><mailto:<email address hidden>>>; Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>; Ignatious Johnson <<email address hidden><mailto:<email address hidden>>>; Ramprakash Ram Mohan <<email address hidden><mailto:<email address hidden>>>
Subject: FW: 5.0.01 /Try Upgrade and Documentation - Working Session



+Linh, Ali, Chaitanya



From: Sarin Kizhakkepurayil <<email address hidden><mailto:<email address hidden>>>
Date: Friday, September 7, 2018 at 9:10 AM
To: Ashutosh Gupta <<email address hidden><mailto:<email address hidden>>>, Ignatious Johnson <<email address hidden><mailto:<email address hidden>>>, Ramprakash Ram Mohan <<email address hidden><mailto:<email address hidden>>>
Cc: Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>
Subject: Re: 5.0.01 /Try Upgrade and Documentation - Working Session



+Ignatious, Ram



  1.  UI works based on keystone token expiry. Extending default expiry on keystone should work (UI team has not tested this)
  2.  From UI setup wizard, under advanced options in orchestrator step, you could set kolla globals key value params. but I believe sending second level key value will not work. Ram/Ignatious can confirm this.
  3.  This needs some more investigation. We ask nova for the console URL and redirect user to same. There may be gap in base URL host:port as we’ve not tested with changing that. I’ll get back to you on this



Thanks



From: Ashutosh Gupta <<email address hidden><mailto:<email address hidden>>>
Date: Friday, September 7, 2018 at 8:47 AM
To: Sarin Kizhakkepurayil <<email address hidden><mailto:<email address hidden>>>
Cc: Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>
Subject: FW: 5.0.01 /Try Upgrade and Documentation - Working Session



Hello Sarin



Marketing team has some queries regarding Contrail Command Setup which is mentioned in the trailing email. Can you please answer some of the queries?



Thanks

Ashutosh





From: Linh Phan <<email address hidden><mailto:<email address hidden>>>
Date: Thursday, September 6, 2018 at 9:48 AM
To: Ali Zaringhalam <<email address hidden><mailto:<email address hidden>>>, Ashutosh Gupta <<email address hidden><mailto:<email address hidden>>>, Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>
Cc: Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>
Subject: RE: 5.0.01 /Try Upgrade and Documentation - Working Session



Hi Ashutosh,



I have some questions regards to Contrail Command setup.



  1.  How do I change/increase the timeout for Horizon/Contrail GUI ? Many users have complained about the GUI keep timing out while they’re doing demo.



  1.  Is there an easy way to add customized “nova.conf” params when installing with Contrail Command ?

For example, if manually install with ansible-deployer, the instances.yaml have this section. The only way I can do it now is by manually add those line in all nova.conf

kolla_config:

  openstack_release: ocata

  customize:

    nova.conf: |

      [oslo_messaging_notifications]

      driver = messagingv2



      [DEFAULT]

      disk_allocation_ratio = 20.0

      ram_allocation_ratio = 5.0

      cpu_allocation_ratio = 25.0





  1.  This VNC params in nova.conf is not taking effects. This makes it not possible to directly access instance’s VNC console, from both Horizon/ContrailCommand.

      [vnc]

      enabled = True

      novncproxy_base_url = http://127.0.0.1:20002/vnc_auto.html



In details:

  *   For the current manual setup with 5.0.1, if I include the above params, I can directly access instance console from my web browser, by port map  192.168.250.1:6080 (remote VNC port) to 127.0.0.1:20002 (local)

(Note the URL for console is 127.0.0.1:20002, straight from Horizon)

[id:image006.png@01D45457.A88678E0]



But if I use Contrail Command, with the same params in nova.conf, it not taking effect, and console_vnc_url still pointing to 192.168.250.1:6080 and timeout. This make direct console access not possible (from both OpenStack and Contrail Command)

[id:image007.png@01D45457.A88678E0]



If I manually change the above URL to 127.0.0.1:20002, console works. So this novncproxy_base_url param under /etc/kolla/*/nova.conf is not in effect. I reboot the hosts several time already.

[id:image008.png@01D45457.A88678E0]







How should I resolve those issues?



Thank you.



Regards,

Linh





From: Ali Zaringhalam
Sent: Tuesday, September 4, 2018 3:13 PM
To: Ashutosh Gupta <<email address hidden><mailto:<email address hidden>>>; Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>; Linh Phan <<email address hidden><mailto:<email address hidden>>>
Cc: Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>
Subject: RE: 5.0.01 /Try Upgrade and Documentation - Working Session



Hi Ashutosh:



Thanks for your investigation.  We will increase memory to 64 GB and test again to see if we see the issue. However I would like to have some clarification regarding memory as the root cause.  In 3.2.3 we had 24 G of RAM and it served us quite well with all aspects of the setup including Cassandra.  While 5.0.1 as a whole may require more memory (we increased it to 32 G and still have some headroom in “top”), it is not clear why Cassandra by itself requires so much more RAM in 5.0.1, and then only intermittently?



In 3.2.3 we ran 5 PODs on a single server and are able to support 30 PODs on 6 servers.  In 5.0.1 we reduced the count to 4 (and a total of 6x4=24 PODs) because of higher memory requirements and the addition of a second compute node.  If we are required to assign 64 G to the config node alone, we will have to reduce the count still further to 3 PODs per server.  With our 6 servers currently in place, the number of PODs will be reduced from the current 30 in 3.2.3 to 18 in 5.0.1.  Given the significant resource impact I wonder if there are parameters that can be tweaked to manage this better for a non-production environment such as ours?



Regards,



Ali



From: Ashutosh Gupta
Sent: Tuesday, September 4, 2018 4:27 PM
To: Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>; Linh Phan <<email address hidden><mailto:<email address hidden>>>
Cc: Ali Zaringhalam <<email address hidden><mailto:<email address hidden>>>; Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>
Subject: Re: 5.0.01 /Try Upgrade and Documentation - Working Session



Hello Chaitanya



The Cassandra state down issue is resolved and the problem was low memory on the config node. It is advisable to increase the memory from 32 Gb to 64 Gb on the config node to avoid this issue in the future. Also as Sarin suggested you can fresh install Contrail Command 5.0.1.



Thanks,

Ashutosh





________________________________

From: Chaitanya Kadiyala
Sent: Friday, August 31, 2018 12:35:04 PM
To: Dilip Sundarraj; Ashutosh Gupta; Arohi Gupta; Linh Phan
Cc: Ali Zaringhalam
Subject: 5.0.01 /Try Upgrade and Documentation - Working Session
When: Tuesday, September 4, 2018 11:00 AM-12:00 PM.
Where: Skype Meeting



Hi Ali,



Dilip and his team are on-board in helping with expediting the documentation for 5.0.1 tasks for the sandbox as well as help with debugging the upgrade including COntrial Command.



I want to bring his team and yours together in this call to address any potential gaps in the existing upgrade. Hope the timing works for you all.



Thanks
Chaitanya



.........................................................................................................................................

Join Skype Meeting<https://meet.juniper.net/ckadiyala/648LHH93>

Trouble Joining? Try Skype Web App<https://meet.juniper.net/ckadiyala/648LHH93?sl=1>

Join by phone



+14089369000<tel:+14089369000> (USA, West Coast)                   English (United States)

+19785898300<tel:+19785898300> (USA, West Coast)                   English (United States)

ILYNC (45962)<tel:ILYNC%20(45962)> (USA, West Coast)                     English (United States)

+18446454398<tel:+18446454398> (USA, West Coast)                   English (United States)

+18332135669<tel:+18332135669> (USA, West Coast)                   English (United States)



Find a local number<https://dialin.juniper.net>



Conference ID: 58410349

 Forgot your dial-in PIN?<https://dialin.juniper.net> |Help<https://o15.officeredir.microsoft.com/r/rlidLync15?clid=1033&p1=5&p2=2009>





Please consider with whom you are communicating, including non-Juniper personnel of companies federated on Skype for Business, before sharing any confidential information.  Non-disclosure agreements may apply.

[!OC([1033])!]

.........................................................................................................................................





_____________________________________________
From: Chaitanya Kadiyala
Sent: Friday, August 31, 2018 12:26 PM
To: Dilip Sundarraj <<email address hidden><mailto:<email address hidden>>>; Ashutosh Gupta <<email address hidden><mailto:<email address hidden>>>; Arohi Gupta <<email address hidden><mailto:<email address hidden>>>
Cc: Yusuke Kanamori <<email address hidden><mailto:<email address hidden>>>; Ali Zaringhalam <<email address hidden><mailto:<email address hidden>>>; Jerry Passione <<email address hidden><mailto:<email address hidden>>>
Subject: FW: 5.0.01 /Try pod 26 to 30 SSH keys



Hi Dilip, Ashutosh, Arohi,



We have 5.0.1 installed with the following infrastructure.

•         2 compute nodes to support VM migration.

•         JTI telemetry from the vMX gateway to AppFormix collector. Traffic from the overlay VMs (outbound towards internet; to/from linux jump server with floating IP) can be viewed in AppFormix dashboard for vMX.

•         ssh port-forwarding through the Linux jump server.  GUI access via local browser.  (RDP will no longer be supported.)

•         As before, full support for floating IP with outbound internet access from overlay.  ssh access to overlay from the Linux jump server.

•         All legacy images are included.  In addition, a vSRX 15.1X49-D140.2 (in factory default) and Kali Linux images are available.

You can logon to Pod 28 (access keys listed in the attached email) to see the current build and stability of the setup.



We do need help from your team in expediting the Contrail Command install as well as in updating the documentation for the sandbox.



I’ve cc’ed Ali from OpenLab who is driving the upgrade and facilitating the infra for the sandbox.



I will setup a call between all of us for Tuesday to go over the details for this sandbox upgrade.



Thanks
Chaitanya



From: Laurent Paumelle
Sent: Thursday, August 23, 2018 11:33 AM
To: Ali Zaringhalam <<email address hidden><mailto:<email address hidden>>>; Stuart Mackie <<email address hidden><mailto:<email address hidden>>>; Linh Phan <<email address hidden><mailto:<email address hidden>>>; Fredric Linder <<email address hidden><mailto:<email address hidden>>>; Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>; Chameera Wijesinghe <<email address hidden><mailto:<email address hidden>>>
Cc: Scott Alexander <<email address hidden><mailto:<email address hidden>>>; Jerry Passione <<email address hidden><mailto:<email address hidden>>>; Marcelino Chua <<email address hidden><mailto:<email address hidden>>>; Maria Caraballo <<email address hidden><mailto:<email address hidden>>>
Subject: Re: 5.0.01 /Try pod 26 to 30 SSH keys



That’s fine for me ☺ it was just a generic question as other people may also need to put their own software sometimes on a pod.

Thanks for the setup!

Laurent



From: Ali Zaringhalam <<email address hidden><mailto:<email address hidden>>>
Date: Thursday 23 August 2018 at 10:14
To: Laurent Paumelle <<email address hidden><mailto:<email address hidden>>>, Stuart Mackie <<email address hidden><mailto:<email address hidden>>>, Linh Phan <<email address hidden><mailto:<email address hidden>>>, Fredric Linder <<email address hidden><mailto:<email address hidden>>>, Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>, Chameera Wijesinghe <<email address hidden><mailto:<email address hidden>>>
Cc: Scott Alexander <<email address hidden><mailto:<email address hidden>>>, Jerry Passione <<email address hidden><mailto:<email address hidden>>>, Marcelino Chua <<email address hidden><mailto:<email address hidden>>>, Maria Caraballo <<email address hidden><mailto:<email address hidden>>>
Subject: RE: 5.0.01 /Try pod 26 to 30 SSH keys



We should consider crx when we build k8/contrail.  otherwise it is obsolete.



Regards,



Ali



From: Laurent Paumelle
Sent: Thursday, August 23, 2018 12:45 PM
To: Ali Zaringhalam <<email address hidden><mailto:<email address hidden>>>; Stuart Mackie <<email address hidden><mailto:<email address hidden>>>; Linh Phan <<email address hidden><mailto:<email address hidden>>>; Fredric Linder <<email address hidden><mailto:<email address hidden>>>; Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>; Chameera Wijesinghe <<email address hidden><mailto:<email address hidden>>>
Cc: Scott Alexander <<email address hidden><mailto:<email address hidden>>>; Jerry Passione <<email address hidden><mailto:<email address hidden>>>; Marcelino Chua <<email address hidden><mailto:<email address hidden>>>; Maria Caraballo <<email address hidden><mailto:<email address hidden>>>
Subject: Re: 5.0.01 /Try pod 26 to 30 SSH keys



Actually there WAS a plugin at the Contrail 3.x timeframe named nova-docker for Openstack. With this any container could be launched directly like a regular VM in the UI/Heat template.



Later with Contrail 4.x I used instead a script to integrate and launch cSRX and create the interface mapping to the vRouter. I do not remember yet but I think the Nova-Docker was already deprecated and no replacement ready. Anyway that script was made by a Contrail developer and helped a lot.



Now with Contrail 5.x and newer Openstack, I’ll need to look whether a plugin exists to integrate with containers but I believe that yes we may need to install Contrail/K8 in that case.

cSRX does not work with K8 per say as it would then need “CNN” (multiple interface support as opposition to CNI for single interface) but it does work thanks to Contrail vRouter since it’s mapped to it rather than to the native K8 networking and Contrail does the service chaining between vnetworks with cSRX or any VNF of course. It does solve this puzzle ☺



Laurent



From: Ali Zaringhalam <<email address hidden><mailto:<email address hidden>>>
Date: Thursday 23 August 2018 at 09:20
To: Laurent Paumelle <<email address hidden><mailto:<email address hidden>>>, Stuart Mackie <<email address hidden><mailto:<email address hidden>>>, Linh Phan <<email address hidden><mailto:<email address hidden>>>, Fredric Linder <<email address hidden><mailto:<email address hidden>>>, Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>, Chameera Wijesinghe <<email address hidden><mailto:<email address hidden>>>
Cc: Scott Alexander <<email address hidden><mailto:<email address hidden>>>, Jerry Passione <<email address hidden><mailto:<email address hidden>>>, Marcelino Chua <<email address hidden><mailto:<email address hidden>>>, Maria Caraballo <<email address hidden><mailto:<email address hidden>>>
Subject: RE: 5.0.01 /Try pod 26 to 30 SSH keys



Not sure if there is “and may be some docker plugin for Openstack/Contrail”.  cSRX should probably be done with K8/Contrail.



Regards,



Ali



From: Laurent Paumelle
Sent: Thursday, August 23, 2018 12:17 PM
To: Ali Zaringhalam <<email address hidden><mailto:<email address hidden>>>; Stuart Mackie <<email address hidden><mailto:<email address hidden>>>; Linh Phan <<email address hidden><mailto:<email address hidden>>>; Fredric Linder <<email address hidden><mailto:<email address hidden>>>; Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>; Chameera Wijesinghe <<email address hidden><mailto:<email address hidden>>>
Cc: Scott Alexander <<email address hidden><mailto:<email address hidden>>>; Jerry Passione <<email address hidden><mailto:<email address hidden>>>; Marcelino Chua <<email address hidden><mailto:<email address hidden>>>; Maria Caraballo <<email address hidden><mailto:<email address hidden>>>
Subject: Re: 5.0.01 /Try pod 26 to 30 SSH keys



Thanks Ali and Linh for the setup.



Quick question: how complex is it to customize a Pod with additional software?

For example, I’m also planning to test with cSRX at some point and then will need Docker (and may be some docker plugin for Openstack/Contrail) and then I may need to try adding some additional components, but I do not want to break things of course. Then my question on how complex it is and if possible at all?



Thanks

Laurent



From: Ali Zaringhalam <<email address hidden><mailto:<email address hidden>>>
Date: Thursday 23 August 2018 at 08:17
To: Stuart Mackie <<email address hidden><mailto:<email address hidden>>>, Linh Phan <<email address hidden><mailto:<email address hidden>>>, Fredric Linder <<email address hidden><mailto:<email address hidden>>>, Laurent Paumelle <<email address hidden><mailto:<email address hidden>>>, Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>, Chameera Wijesinghe <<email address hidden><mailto:<email address hidden>>>
Cc: Scott Alexander <<email address hidden><mailto:<email address hidden>>>, Jerry Passione <<email address hidden><mailto:<email address hidden>>>, Marcelino Chua <<email address hidden><mailto:<email address hidden>>>, Maria Caraballo <<email address hidden><mailto:<email address hidden>>>
Subject: RE: 5.0.01 /Try pod 26 to 30 SSH keys



•         Based on feedback from folks on this thread we need to install Contrail Command.

•         contrail-command and contrail-command-deployer are available from the GA registry but with a different tag (simple 5.0.1-0.214 instead of 5.0.1-0.214-ocata).  We don’t need to go to a separate registry.  It needs a separate installation.

•         We will install Contrail Command on the Linux jump server.  This obviates the need for a separate VM to host Contrail Command.  It will be port-mapped.



Regards,



Ali



From: Stuart Mackie
Sent: Thursday, August 23, 2018 1:44 AM
To: Linh Phan <<email address hidden><mailto:<email address hidden>>>; Ali Zaringhalam <<email address hidden><mailto:<email address hidden>>>; Fredric Linder <<email address hidden><mailto:<email address hidden>>>; Laurent Paumelle <<email address hidden><mailto:<email address hidden>>>; Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>; Chameera Wijesinghe <<email address hidden><mailto:<email address hidden>>>
Cc: Scott Alexander <<email address hidden><mailto:<email address hidden>>>; Jerry Passione <<email address hidden><mailto:<email address hidden>>>; Marcelino Chua <<email address hidden><mailto:<email address hidden>>>; Maria Caraballo <<email address hidden><mailto:<email address hidden>>>
Subject: Re: 5.0.01 /Try pod 26 to 30 SSH keys



Definitely, it will be the default GUI for most configuration actions.



Stuart

-914 886 2534



From: Linh Phan <<email address hidden><mailto:<email address hidden>>>
Date: Thursday, August 23, 2018 at 7:20 AM
To: Stuart Mackie <<email address hidden><mailto:<email address hidden>>>, Ali Zaringhalam <<email address hidden><mailto:<email address hidden>>>, Fredric Linder <<email address hidden><mailto:<email address hidden>>>, Laurent Paumelle <<email address hidden><mailto:<email address hidden>>>, Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>, Chameera Wijesinghe <<email address hidden><mailto:<email address hidden>>>
Cc: Scott Alexander <<email address hidden><mailto:<email address hidden>>>, Jerry Passione <<email address hidden><mailto:<email address hidden>>>, Marcelino Chua <<email address hidden><mailto:<email address hidden>>>, Maria Caraballo <<email address hidden><mailto:<email address hidden>>>
Subject: RE: 5.0.01 /Try pod 26 to 30 SSH keys



Hi Stuart,



We use the standard Contrail-ansible-deployer to deploy this setup:

https://github.com/Juniper/contrail-ansible-deployer



Contrail-Command-Deployer is a separate setup and is not included with the standard deployer. I checked port 9091 (no response) and no Contrail-Command container is present.

https://github.com/Juniper/contrail-command-deployer



We will need to experiment and test Contrail-Command on a separate setup. Do you suggest we put Contrail-Command for the /try as well?



Regards,

Linh.

From: Stuart Mackie
Sent: Wednesday, August 22, 2018 10:02 PM
To: Ali Zaringhalam <<email address hidden><mailto:<email address hidden>>>; Fredric Linder <<email address hidden><mailto:<email address hidden>>>; Laurent Paumelle <<email address hidden><mailto:<email address hidden>>>; Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>; Chameera Wijesinghe <<email address hidden><mailto:<email address hidden>>>
Cc: Scott Alexander <<email address hidden><mailto:<email address hidden>>>; Jerry Passione <<email address hidden><mailto:<email address hidden>>>; Marcelino Chua <<email address hidden><mailto:<email address hidden>>>; Maria Caraballo <<email address hidden><mailto:<email address hidden>>>; Linh Phan <<email address hidden><mailto:<email address hidden>>>
Subject: Re: 5.0.01 /Try pod 26 to 30 SSH keys



Is there a port map for Contrail Command?



Stuart

-914 886 2534



From: Ali Zaringhalam <<email address hidden><mailto:<email address hidden>>>
Date: Thursday, August 23, 2018 at 4:13 AM
To: Stuart Mackie <<email address hidden><mailto:<email address hidden>>>, Fredric Linder <<email address hidden><mailto:<email address hidden>>>, Laurent Paumelle <<email address hidden><mailto:<email address hidden>>>, Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>, Chameera Wijesinghe <<email address hidden><mailto:<email address hidden>>>
Cc: Scott Alexander <<email address hidden><mailto:<email address hidden>>>, Jerry Passione <<email address hidden><mailto:<email address hidden>>>, Marcelino Chua <<email address hidden><mailto:<email address hidden>>>, Maria Caraballo <<email address hidden><mailto:<email address hidden>>>, Linh Phan <<email address hidden><mailto:<email address hidden>>>
Subject: 5.0.01 /Try pod 26 to 30 SSH keys



Hi Folks:



The 5.0.1 release is now available for testing.  You can use the same ssh keys we sent earlier for the 5.0 setups.



1.       POD 27 (DC-198): Stuart Mackie

2.       POD 28 (DC-204): Chaitanya Kadiyala

3.       POD 29 (DC-210): Fred Linder/Chameera Wijesinghe



@Laurent: we are setting you up on another server (POD25) and will send you the coordinates.



Please note that due to the increase in the number of compute nodes, we can only fit 4 PODs per ESXi server.  Given the 6 servers at our disposal, we can only support 24 PODs instead of the previous 30.  We are working to increase the number of servers.



Regards,



Ali



_____________________________________________
From: Ali Zaringhalam
Sent: Monday, July 9, 2018 11:13 PM
To: Stuart Mackie <<email address hidden><mailto:<email address hidden>>>; Fredric Linder <<email address hidden><mailto:<email address hidden>>>; Laurent Paumelle <<email address hidden><mailto:<email address hidden>>>; Chaitanya Kadiyala <<email address hidden><mailto:<email address hidden>>>; Chameera Wijesinghe <<email address hidden><mailto:<email address hidden>>>
Cc: Scott Alexander <<email address hidden><mailto:<email address hidden>>>; Jerry Passione (<email address hidden><mailto:<email address hidden>>) <<email address hidden><mailto:<email address hidden>>>; Marcelino Chua <<email address hidden><mailto:<email address hidden>>>; Maria Caraballo <<email address hidden><mailto:<email address hidden>>>; Linh Phan <<email address hidden><mailto:<email address hidden>>>
Subject: FW: /Try pod 26 to 30 SSH keys





Hi Folks:



The Contrail 5.0 sandboxes are now available for testing in the JCL /try.  The primary purpose of making these available early is to test existing use cases and user experience.  The target general availability in all 30 sandboxes is 8/15.  This is contingent on incorporating your early feedback and the availability of user documentation for both Contrail and AppFormix.



The sandboxes are allocated as follow:



1.       POD 26 (DC-192): Linh Phan

2.       POD 27 (DC-198): Stuart Mackie

3.       POD 28 (DC-204): Chaitanya Kadiyala

4.       POD 29 (DC-210): Fred Linder/Chameera Wijesinghe

5.       POD 30 (DC-216): Laurent Paumelle



The following new capabilities are available:



•         2 compute nodes to support VM migration.

•         JTI telemetry from the vMX gateway to AppFormix collector. Traffic from the overlay VMs (outbound towards internet; to/from linux jump server with floating IP) can be viewed in AppFormix dashboard for vMX.

•         ssh port-forwarding through the Linux jump server.  GUI access via local browser.  (RDP will no longer be supported.)

•         As before, full support for floating IP with outbound internet access from overlay.  ssh access to overlay from the Linux jump server.

•         All legacy images are included.  In addition, a vSRX 15.1X49-D140.2 (in factory default) and Kali Linux images are available.



The intent of the sandbox is to provide a platform to support OpenStack/Contrail/AppFormix features.  The primary target users are external reservations and Contrail/AppFormix training classes.  Users/trainers are responsible for the development and testing of their use cases.  We plan to provide similar sandboxes with expanded capabilities and privileges for internal COE/SE users in JCL BYO in September.   Until that time, the use of these sandboxes for external demos should be considered as temporary.



Let us know if you have any questions.



Regards,



Ali





_____________________________________________
From: Linh Phan
Sent: Monday, July 9, 2018 6:34 PM
To: Ali Zaringhalam <<email address hidden><mailto:<email address hidden>>>
Subject: /Try pod 26 to 30 SSH keys





Hi Ali,



In attachment is the zip that contains pod 26-30 keys plus corresponding *.bat and *.sh file, and a README file (Please let me know if it’s missing anything).



Below is a brief instruction, please feel free to copy it. Everything works, except the Alarms still appears (I deleted few days ago, they go back on).

<< File: keys.zip >>



Pod 27 – DC-198

Pod 28 – DC-204

Pod 29 – DC-210

Pod 30 – DC-216



Enter URL on laptop browser: ( no need http:// or https:// )

Horizon: 127.0.0.1:20000

Contrail: 127.0.0.1:20001

AppFormix: 127.0.0.1:20003



Credentials:

OpenStack/Contrail/AppFormix : admin | contrail123       (domain: default)

Cirros: cirros | gocubsgo

Ubuntu: contrail | contrail123

vSRX: root | contrail123





Regards,

Linh