Ruby Proxies for Scale,

Performance and Monitoring

Ilya Grigorik
CTO / AideRSS

Ruby Proxies +Architectures

Event-Driven EventMachine http://bit.ly/em-proxy
http://bit.ly/eda @igrigorik
@igrigorik#gogaruco
#meshu #eda
 postrank.com/topic/ruby

The slides… Twitter My blog

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Proxy Love

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 “Rails, Django, Seaside, Scala…” cant scale.

Myth: Slow Frameworks

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 The Proxy Solution

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 The “More” Proxy Solution

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Transparent
HAProxy

App server A App server B

Transparent, Cut-Through Proxy

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 90% use case

Transparent Proxy = Scalability Power Tool

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Proxies for Monitoring, Performance and Scale
welcome to the wonderful world of intercepting, caching, … , proxy servers

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Simulated User

Proxy Proxy

App server A App server B App server C

Production Problem: Staging Environment

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Simulating traffic?

Proxy

Duplication

App server C

“Representative Load / Staging”

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Profile of queries has changed Fail
Load on production has changed Fail
Parallel environment Fail
Slower release cycle Fail

Staging fail.

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 github.com/igrigorik/autoperf

Replay log data, rinse, repeat

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Duplex Ruby Proxy, FTW!

Production

Real (production) traffic

Benchmark

Benchmarking Proxy
flash of the obvious

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Transparent Duplex Proxy, FTW

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Same response, different
turnaround time

Different response body!

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Duplex Ruby Proxy, FTW!
github.com/igrigorik/em-proxy

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 1. Accept connection
- Connect & Forward to Production
- Connect & Forward to Staging

2. Stream response to client from Production

- Buffer response from Staging

3. Run “Processor / Analyzer” once both return

EM-Proxy Workflow

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 EM::Connection

#receive_data
- new data ready for processing

#unbind
- connection terminated

#connection_completed
- connected

Ruby EventMachine
crash course

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Duplex incoming data

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Forward Production
Response

Analyze Results

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Open connection to
application server

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Buffer until connected

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Pass response to Duplex

Notify when done

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Analyze Time, Response…

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Analyze Time, Response…

Response Time

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Cute, right?
but so what…

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 %w[ <Transparent> Intercepting Caching … ]
solution for every problem

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Woops!

Validating Proxy
easy, real-time diagnostics

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Who said you’re not allowed to touch the data?

Intercepting Proxy
for those hard-to-reach spots

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 SMTP / Mail: inject SPAM detection
De(encrypt): stream data
MySQL: inject sharding logic

Intercepting Proxy
for those hard-to-reach spots

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 @PostRank: Beanstalkd + Ruby Proxy
because RAM is still expensive

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 ~ 93 Bytes of overhead per job
~300 Bytes of data / job

x 80,000,000 jobs in memory

~ 30 GB of RAM = 2 X-Large EC2 instances

Oi, expensive!

Beanstalkd Math

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Observations:
1. Each job is rescheduled several times
2. > 95% are scheduled for > 3 hours into the future

Memory is wasted…

3. Beanstalkd does not have overflow page-to-disk

We’ll add it ourselves!

Extending Beanstalkd

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Intercepting Proxy 1 “Medium” EC2 Instance

MySQL
EM-Proxy

Beanstalkd

@PostRank: “Chronos Scheduler”

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Extend the protocol

Process incoming data

Buffer “SCHEDULE” requests, otherwise forward

Chronos Scheduler
Beanstalkd persistence

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Buffer “SCHEDULE”

Archive to MySQL

Chronos Scheduler
Beanstalkd persistence

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Extend the protocol

MySQL
EM-Proxy

ARCHIVE

10 line patch to client: RESERVE, …

queue.archive(job)
Beanstalkd

@PostRank: “Chronos Scheduler”

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 400% cheaper + extensible! ~79,000,000 jobs, 4GB RAM

MySQL
EM-Proxy

ARCHIVE

Upcoming jobs: ~ 1M
RESERVE, …
1 Proxy, 1 Beanstalkd, 1 Beanstalkd
MySQL = 200 req/s

@PostRank: “Chronos Scheduler”

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 %w[ Transparent <Intercepting> Auth Cache … ]

easy to make & a powerful hammer

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco

 Cheers. Questions?

The slides… Twitter My blog

Ruby Proxies + EventMachine http://bit.ly/em-proxy @igrigorik #gogaruco