Loading…
MesosCon NA 2016 has ended
avatar for Michael Payne

Michael Payne

JPMorgan Chase
Executive Director
michael.h.payne@jpmchase.com
Michael Payne is an engineer at JPMorgan Chase where he leads the Kubernetes Architecture team. He works with the Envoy community particularly in the areas of dependency management and supply chain. His Envoy interests include new protocols (UDP, HTTP/3), load balancing and egress patterns.
Wednesday, June 1
 

9:00am PDT

Keynote: Welcome Address - Benjamin Hindman, Founder and Chief Architect, Mesosphere
Wednesday June 1, 2016 9:00am - 9:30am PDT
Grand Mesa A-F
  Keynote

9:30am PDT

Keynote: The New Open Source Data Center - Joshua Bernstein, Vice President of Technology, EMC {code}
In today’s ever shifting world the most accomplished CIO’s know that adaptability is crucial for their organizations success. For the past 20 years some of the most successful organizations in the world have been using open source platforms and tools to break the status quo and disrupt industries. EMC {codes}’s Josh Bernstein will share his personal experiences doing just that from his days running the Siri Deployment and Infrastructure Architecture team at Apple. By embracing open source and DevOps tools and methodologies, Josh’s team was able to take  Siri from launch to tens of thousands of servers, deployed in more than a dozen locations, in under 5 years.  Join us to find out why building open source data centers is the key to the modern data center and how the world’s largest organizations are taking advantage of this knowledge.

The New Open Data Center pdf
Wednesday June 1, 2016 9:30am - 9:40am PDT
Grand Mesa A-F
  Keynote

9:45am PDT

Keynote: A Real-Time Cloud for the Internet of Things - Professor Ken Birman, Cornell University
We’ve all read the hype about the Internet of Things: a technology trend that seems endlessly about to happen, yet has been stubbornly hard to integrate with cloud computing.  In this talk I want to ask why this has been so, and how we can fix it.   

I’ll start by describing work Cornell has done over the past few years on creating a cloud platform to host “smart power grid” applications.  This involves (1) new rack-scale management solutions aimed at applications that need to run 24x7, (2) replication with ultra-fast updates for scalable real-time responsiveness,  and (3) new real-time storage solutions, to enable a new kind of big-data temporal computing.  But new models also create huge performance puzzles.  For our work, these center on how to overcome Brewer’s CAP principle.  CAP was about performance tradeoffs, and the key to conquering CAP is to leverage RDMA and NVRAM hardware, which offer amazing speedups for critical data paths.   By working from the ground up, and using RDMA and NVRAM as accelerators for data replication, storage and management solutions, we can get huge speedups compared to older styles of cloud computing, even when consistency and fault-tolerance are required.

This work is all open-source, and should be of special interest to the Apache Mesos community because container models turn out to be one of the key enablers for pulling off these tricks.  With older styles of cloud virtualization, it can be incredibly hard to offer scheduling guarantees, and technologies like RDMA and NVRAM are very hard to virtualize.  Containers eliminate both of those obstacles. 

Ken Birman pdf
Wednesday June 1, 2016 9:45am - 10:15am PDT
Grand Mesa A-F
  Keynote

10:15am PDT

Keynote: Verizon Calls Mesos - Larry Rau, Director of Architecture and Infrastructure, Verizon Labs
As data services change the way the world does business, Verizon Labs has built a platform designed around the Mesos open source system that enables the robust development of micro services for a variety of products and services. This presentation will use a real world example of how Verizon Lab's Mesos-based platform integrates with America's most reliable wireless network to transform how Verizon builds and delivers new services.
Wednesday June 1, 2016 10:15am - 10:25am PDT
Grand Mesa A-F
  Keynote

10:30am PDT

Coffee Break and Sponsor Showcase
Wednesday June 1, 2016 10:30am - 11:00am PDT
Grand Mesa Foyer
  Breaks

11:00am PDT

Containers in Apache Mesos: Present and Future - Jie Yu & Tim Chen, Mesosphere
Mesos currently supports two types of container runtimes: one is Mesos’ own pluggable runtime which uses OS primitives directly (a.k.a. the Mesos containerizer); the other is to delegate the entire container management to Docker engine (a.k.a. the Docker containerizer). Recently, we added container image support to Mesos containerizer, which allows it to launch off-the-shelf Docker or Appc containers without depending on Docker daemon or rkt. In this talk, we are going to discuss the motivation of why we want to unify the containerizers in Mesos and how this benefits the Mesos maintainers and our users. We will then talk about the current status of this work, and show what can be built on top of this pluggable architecture. Finally, we will share our vision on the near future for the Mesos containerizer and how it’s going to enable more innovation in the Mesos ecosystem.
MesosCon'16 Container in Mesos Present and Future. pdf
Wednesday June 1, 2016 11:00am - 11:50am PDT
Grand Mesa DE
  Developer, Intermediate

12:00pm PDT

Apache Mesos on Windows: Status and the Roadmap - Alex Clemmer & Daniel Pravat, Microsoft
Windows Server 2016 is expected to be made generally available sometime in Q3 2016, and with it, the Windows Containers API.

Supporting Windows Containers natively is a major growth opportunity for the Mesos community -- by enabling users to operate clusters containing a mix of Unix and Windows hosts, it is possible to significantly reduce administrative overhead and increase resource utilization in data centers.

In this talk we will provide a ground-up view of the work we have done over the last year to make the Windows agent a reality.

We’ll cover the changes made to Mesos core so far, discussing in detail how the Mesos agent works on Windows. We will then discuss the implications of these changes at the service level, covering how to operate clusters with a mix of Windows and Unix nodes in practice.

Finally, we will present a demo running on such a cluster, and a roadmap for future plans.
Wednesday June 1, 2016 12:00pm - 12:50pm PDT
Grand Mesa ABC
  Developer, Beginner

1:20pm PDT

Lightning Talk: An off-the-network approach to monitoring microservices - Sahil Bansal, Netsil

The increased adoption of microservices has necessitated a rethink in the way application performance monitoring works for distributed systems. We propose an off-the-network, stream processing approach to create a real-time copy of application chatter and use this copy as the “source of truth” to create a topology map of services. This topology map helps SREs to visually identify choke-points and hotspots and identifying API end-points makes it a completely framework agnostic approach reducing the overall time to value for performance monitoring and addressing SLA issues.

Wednesday June 1, 2016 1:20pm - 1:25pm PDT
Grand Mesa A-F
  Lightning Talks

2:00pm PDT

Throw Away Your Firewalls and Make Your Applications More Secure - Casey Davenport, Project Calico
Count the number of data breaches in 2015 and it's clear that enterprise security is splitting at the seams. The old model of trying to secure large applications using firewall appliances between each application “tier” just isn't working. As we move into the cloud-native era and break applications into smaller more easily understandable micro-services, there is a huge opportunity to revolutionize security by rendering developer intent directly into the network fabric to create networks that are more secure and more dynamic than ever before. Casey will discuss how evolving micro-services architectures allow you to throw away your traditional firewalls, and transition data center networks from statically configured, application-specific monoliths to highly dynamic, scalable, and secure systems.
MesosCon2016Slides pdf
Wednesday June 1, 2016 2:00pm - 2:50pm PDT
Grand Mesa F
  Operations, Intermediate

3:00pm PDT

Running Cassandra on Apache Mesos Across Multiple Datacenters at Uber - Abhishek Verma & Matthias Eichstaedt & Teng Xu, Uber
Traditionally, machines were statically partitioned across the different services at Uber. In an effort to increase the machine utilization, Uber has recently started transitioning most of its services, including the storage services, to run on top of Mesos. This presentation will describe the initial experience building and operating a framework for running Cassandra on top of Mesos running across multiple datacenters at Uber. This framework automates several Cassandra operations such as node repairs, addition of new nodes and backup/restore. It improves efficiency by co-locating CPU-intensive services as well as multiple Cassandra nodes on the same Mesos agent. It handles failure and restart of Mesos agents by using persistent volumes and dynamic reservations. This talk includes statistics about the number of Cassandra clusters in production, time taken to start a new cluster, add a new node, detect a node failure; and the observed Cassandra query throughput and latency.
mesoscon uber pdf
Wednesday June 1, 2016 3:00pm - 3:50pm PDT
Grand Mesa ABC
  Developer, Any

3:00pm PDT

Apache Mesos Security Best Practices (2016) - Adam Bordelon & Alexander Rojas, Mesosphere
Organizations migrating their infrastructure to Apache Mesos are concerned about protecting their critical workloads and sensitive data from naive/untrusted users and programs. Going beyond last year’s tips about encryption, authentication, and authorization, this year’s talk will cover advancements in:
- Authentication and authorization module interfaces
- Authenticating and authorizing HTTP operator endpoints
- Filtering Web UI and state endpoints based on HTTP user
- Protecting sandbox access
Come prepared to learn about the above, and share with us your needs and stories
Wednesday June 1, 2016 3:00pm - 3:50pm PDT
Grand Mesa F
  Operations, Intermediate

4:20pm PDT

A Seamless Monitoring System for Apache Mesos Clusters - Drew Gassaway, Bloomberg LP
Mesos and its distributed processing frameworks offer an easy way to migrate computation tasks to a machine-agnostic environment. To use the capabilities of Mesos to execute distributed tasks effectively a unified monitoring system is required. The monitoring system has to address the integration of disparate logging and metric facilities resulting from the heterogeneous tasks that can be executed on Mesos.

In this talk, we'll look at a framework that offers a way to ingest log, metric, and other data, and send it upstream. We'll go into detail about how the framework uses the dynamic Mesos state available on each slave to aggregate logs and metric data in a way that's transparent to running applications. It also provides a plugin-based component that can automatically ingest slave data such as detailed Mesos task resource usage statistics. We’ll also discuss the performance characteristics.
SeamlessMonitoringForMesosClusters pdf
Wednesday June 1, 2016 4:20pm - 5:10pm PDT
Grand Mesa F
  Operations, Intermediate

5:20pm PDT

Container Network Interface (CNI) Support in Apache Mesos - Jie Yu & Avinash Sridharan, Mesosphere; Qian Zhang, IBM
Currently, despite the wide adoption of Mesos as the kernel for the data center, Mesos does not have a standardized interface to plug into different networks. Existing modules, such as the port mapping isolator or net-modules, are focused solutions that aim to make Mesos work on specific types of networks. In this talk Qian, Avinash and Jie will present the design and implementation of a new network isolator the network/cni isolator that implements the CNI (Container Network Interface) spec, proposed by CoreOS. The network/cni isolator will allow Mesos containers to work, potentially, on any type of IP network. CNI allows for a clean division of work, allowing Mesos to focus on network isolation and offloading network configuration to native binaries called CNI plugins. There are CNI plugins developed for different types of networks (e.g., bridge, flannel, calico, weave etc.) allowing Mesos to run containers on a multitude of different networks.
Container Network Interface (CNI) for Mesos pdf
Wednesday June 1, 2016 5:20pm - 6:10pm PDT
Grand Mesa F
  Operations, Intermediate
 
Thursday, June 2
 

9:00am PDT

Keynote: Day 2 Welcome and Special Demo - Benjamin Hindman, Founder and Chief Architect, Mesosphere
Thursday June 2, 2016 9:00am - 9:15am PDT
Grand Mesa A-F
  Keynote

9:15am PDT

Keynote: Spark 2.0 - Matei Zaharia, Apache Spark Creator and CTO of Databricks
Apache Spark is one of the most widely used systems for big data processing, and it has been integrated with Mesos from day one -- in fact, Spark started as an example framework running on the Mesos cluster manager. In this talk, I'll discuss where Spark fits in the data processing ecosystem, and some exciting additions coming out soon in Spark 2.0. In particular, unlike Apache Hadoop, Spark was designed to be a computing engine agnostic of storage systems, so that it can connect to a variety of systems beyond HDFS. This choice proved useful because most organizations use a mix of storage systems and want to run Spark on elastic infrastructure such as Mesos. Spark 2.0 will go further in this direction through support for new data sources, high-level APIs that can push key parts of the computation into the storage systems, and a new Structured Streaming API that provides similar optimizations for stream processing.
Spark 2.0 pdf
Thursday June 2, 2016 9:15am - 9:45am PDT
Grand Mesa A-F
  Keynote

9:45am PDT

Keynote: Bringing Mesos to the World - Florian Leibert, Founding CEO, Mesosphere
Apache Mesos has powered remarkable innovations inside some of the world’s biggest and smartest technology companies, but not everyone can follow in their footsteps. A more complete package is necessary if we’re going to bring the power of Mesos and all it enables to organizations of every type.
Join Mesosphere CEO Florian Leibert as he introduces DC/OS, an open source platform aimed to do just that. Special guests from Cisco, Time Warner Cable and Autodesk will explain how they’ve used DC/OS to transform their datacenters, take technologies such as containers into production, and rollout new applications in a fraction of the time it would have taken otherwise.
Thursday June 2, 2016 9:45am - 9:55am PDT
Grand Mesa A-F
  Keynote

10:00am PDT

Keynote: Platform Infrastructure at Twitter: The Past, Present and Future - Chris Pinkham, VP of Engineering, Twitter

Platform Engineering builds systems and libraries that enable engineers at Twitter to easily run and operate their services globally at massive scale. These include the core compute and storage infrastructure, search infrastructure, edge, monitoring and messaging platforms, data & analytics platform, and core application services. In recent years, Twitter Inc. has expanded its product portfolio through acquisitions like Periscope, Vine, MoPub, TellApart, Crashlytics, and GNIP of which many of them currently run on the public cloud. To enable deeper integration between products and address the larger business need, it is vital to extend the scope and presence of Twitter’s Platform Infrastructure.

Chris Pinkham, Vice President of Platform Engineering will share the current state of Twitter’s Platform Infrastructure, the challenges when operating at scale and his team’s vision for hybrid cloud services.
Thursday June 2, 2016 10:00am - 10:30am PDT
Grand Mesa A-F
  Keynote

10:30am PDT

Coffee Break and Sponsor Showcase
Thursday June 2, 2016 10:30am - 11:00am PDT
Grand Mesa Foyer
  Breaks

11:00am PDT

It’s Complicated, Okay (or Let’s Talk Openly about Apache Mesos’ OSS Neighbors, Friends and Rivals) - Aaron Williams, Mesosphere
Mesos is never the only OSS you need to run your production datacenter. And just like all of us hanging out together at MesosCon, if you surround Mesos with its OSS friends, you get a happy, highly productive Mesos. But you have to be careful, not all OSS plays well with others.

In this talk, we’ll start by looking at a handful of production Mesos datacenters from major users. We’ll use these real-world examples to abstract a standard Mesos datacenter architecture that contains all of the components needed to run today’s modern, containerized apps with big data and analytics frameworks.

With that harmonized Mesos datacenter architecture, we’ll look at each of the abstract components and discuss the leading OSS projects that fit each piece. We’ll discuss the characteristics of what makes some solutions work well with Mesos, and call out the projects that don’t meet the standard.

We’ll wrap up the talk by showing the complete architecture diagram, and show the single- command trick for bringing all of the best OSS components together and getting them up and running in a production cluster.
Thursday June 2, 2016 11:00am - 11:50am PDT
Grand Mesa F
  Wildcard, Beginner

12:00pm PDT

How We Built a Chargeback System for Twitter’s Compute Platform (Mesos/Aurora) That Incentives Higher Resource Utilization - Micheal Benedict & Jeyappragash Jeyakeerthi, Twitter
Twitter’s compute platform manages hundreds and thousands of containers across tens and thousands of hosts. At that scale, problems we have observed are:
1. Containers are not often sized appropriately in a way to utilize underlying hardware resources efficiently
2. Lack of visibility into who owns what containers (and services)
3. Identifying and tracking the utilization of resources by individual containers to the org structure

We built a Chargeback system for Mesos that provides visibility into who owns what and how much resources they use. It also generates a "bill" based on usage data. The goal here is to incentivize users of Mesos through visibility into what they use & effectively right size their containers to use the underlying resources in a cost-effective way. In this talk, we describe how we built the chargeback service that
1. Ingests utilization and quota data from Mesos master & container stats
2. process around figuring out resources to charge on (ex, cpu cores used per day)
3. ownership mapping i.e how we built a identity service that ties ldap roles with organizational teams (takes a more project/team centric view)

Towards the end of the talk, we will share a case study of how chargeback at Twitter helped improve resource utilization (cores in this case) by over 30% and save millions in $ prior to implementing something like Chargeback.
Thursday June 2, 2016 12:00pm - 12:50pm PDT
Grand Mesa F
  Wildcard, Intermediate

12:50pm PDT

Lunch and Sponsor Showcase
Thursday June 2, 2016 12:50pm - 2:00pm PDT
Grand Mesa Foyer
  Breaks

2:00pm PDT

Monitoring Microservices: Docker, Mesos and DCOS Visibility at Scale - Loris Degioanni, Sysdig
Everyone knows how microservices and containers are revolutionizing the way we deploy applications and maintain our infrastructure. But as many teams are starting to find out, containers still have a key challenge: monitoring and troubleshooting them can be impractical, painful, and sometimes plain impossible. With the rise of microservice based architectures and container orchestration tools such as Mesos and DCOS managing difficult task has become a little bit easier - if you take the right approach.

In this demo-driven presentation, Loris Degioanni will cover the current state of the art of container and microservice monitoring, including the pros and cons of some popular approaches. Using real tools running in live environments, he will demonstrate how to effectively monitor, explore and troubleshoot highly scaled Mesos deployments.

The presentation will feature live interaction with container environments and live demos of all tools and techniques discussed. Special emphasis will be put on using the Mesos portfolio of scheduling and management tools as well as sysdig, an open source container and system troubleshooting tool developed by the presenter.

GitHub link: https://github.com/draios/sysdig

Specific topics will include:
* visualizing the physical vs logical architecture of Mesos & DCOS deployments
* understanding performance at the holistic microservice/application level for orchestrated systems
* Leveraging Mesos metadata such as Master, Slave, Marathon and labels for more intelligent troubleshooting
* extracting process and application-level performance metrics from inside containers using non-intrusive methods
* troubleshooting detailed network activity among distributed containers
Thursday June 2, 2016 2:00pm - 2:50pm PDT
Grand Mesa DE
  Operations, Intermediate

3:00pm PDT

How to Stand Up a 600 Node Bare Metal Mesos Cluster in Two Weeks - Craig Neth, Verizon Labs
In this talk, Craig Neth (Distinguished Member of Technical Staff at Verizon) will describe his experiences in bringing up a 600 node Mesos cluster - from power on to running tasks in 14 days. He will describe the Open Source Hardware and Software he used, as well as the processes and methodology how all these components were used to automate and streamline the process. Neth will also talk about some of the challenges he faced and the improvements he’s hoping to see in the future.
Mesoscon 2016 cneth pdf
Thursday June 2, 2016 3:00pm - 3:50pm PDT
Grand Mesa F
  Wildcard, Any

4:20pm PDT

Migrating 200+ Load Balancers into Apache Mesos - Stephen Salinas, HubSpot
It’s no secret that web services gain enhanced reliability and operational simplicity by running in Mesos. HubSpot wanted to achieve these same benefits for a much more static piece of infrastructure, our load balancers. This talk will explore HubSpot’s open source solution for keeping load balancers reliably reachable and up to date while running them inside a Mesos cluster, and the lessons learned from moving a normally static piece of infrastructure into a dynamic Mesos environment.
SalinasMesosCon2016 pdf
Thursday June 2, 2016 4:20pm - 5:10pm PDT
Chasm Creek
  Operations, Intermediate

5:20pm PDT

Finagle, linkerd, and Apache Mesos: Magical Operability Sprinkles for Microservices - Oliver Gould, Buoyant
Finagle and Mesos are two core technologies used by Twitter and many other companies to scale application infrastructure to high traffic workloads. In this talk, we describe how these two technologies work together to form applications that are both highly scalable and resilient to failure. We introduce linkerd, an open-source proxy form of Finagle, which extends Finagle's operational model to non-JVM or polyglot microservices. Finally, we show how linkerd can be used to "wrap" applications running on Mesos to provide higher-level, service-based semantics around scalability, reliability, and fault-tolerance for microservices---even in the presence of unpredictable traffic volumes and unreliable hardware.
Thursday June 2, 2016 5:20pm - 6:10pm PDT
Chasm Creek
  Developer, Advanced

7:00pm PDT

Offsite Attendee Reception - Viewhouse Centennial

Join fellow attendees for an evening event with a view! ViewHouse is a local chef-driven, casual eatery, bar & rooftop that will provide a great atmosphere to wrap up the event. Get ready for good food, lawn games, and conversation.

Transportation will be provided.


Thursday June 2, 2016 7:00pm - 9:30pm PDT
ViewHouse Centennial 7101 South Clinton Street Centennial, CO 80112
  Evening Events