This is not the document you are looking for? Use the search form below to find more!

Report home > Others

Distributed and concurrent programming with RabbitMQ and EventMachine Rails Underground 2009

0.00 (0 votes)
Document Description
Distributed and concurrent programming with RabbitMQ and EventMachine Rails Underground 2009
File Details
Submitter
  • Name: fazila

We are unable to create an online viewer for this document. Please download the document instead.

Distributed and concurrent programming with RabbitMQ and EventMachine Rails Underground 2009 screenshot
DOWNLOAD

Add New Comment




Related Documents

Introduction to Computing and Programming with Java: A Multimedia Approach, Mark J. Guzdial, Barbara Ericson, PRENTICE HALL, IM+JavaClass+SM+PPT

by: mysmandtb, 9 pages

Solution Manuals and Test Banks I have huge collection of solution manuals and test banks. I strive to provide you unbeatable prices with excellent support. So, I assure you that you won’t be ...

3. Programming with Java Operators and Strings

by: hatem_aziza, 46 pages

3. Programming with Java Operators and Strings

Advanced programming with lcc-win32

by: desantis, 62 pages

C programming with lcc-win32. Jacob Navia and Friedrich Dominicus. This document is part of the lcc-win32 documentation.

Programming with IBM Enterprise PL/I

by: info@test-kings.com, 7 pages

Programming with IBM Enterprise PL/I download from http://www.pass-sure.com is best for IT students.IBM 000-041 exam training gives you a deep insight of the Programming with IBM Enterprise PL/I ...

Introduction to AMQP Messaging with RabbitMQ

by: ludwig, 35 pages

Introduction to AMQP Messaging with RabbitMQ

Cloud Computing And Citrix C3 - July 2009

by: armida, 51 pages

Cloud Computing And Citrix C3 - July 2009

Insurance Finance And Investment July 15 2009 Article On Derivatives Regulation Under Ny Ins Law

by: abelardo, 4 pages

Insurance Finance And Investment July 15 2009 Article On Derivatives Regulation Under Ny Ins Law

GCSE Business and Communication Systems Specification 2009

by: david, 31 pages

GCSE Business and Communication Systems Specification 2009

Gaming and Leisure Magazine, Spring 2009, Thai Spa Assn

by: csenger, 3 pages

Gaming and Leisure Magazine, Spring 2009, Thai Spa Assn

2. Programming with Java Statements

by: hatem_aziza, 40 pages

2. Programming with Java Statements.pdf

Content Preview
  1. “Divide and conquer riding rabbits and trading gems” - a tale about distributed programming - • http://www.slideshare.net/hungryblank • http://github.com/hungryblank/rabbit_starter Paolo Negri @hungryblank
  2. Resources • http://www.slideshare.net/hungryblank • http://github.com/hungryblank/rabbit_starter
  3. About me Time GNU/Linux - Dbs Perl “systems” PHP Python Ruby
  4. Summary:
  5. Distributed Concurrent Programming
  6. rabbitMQ http://www.?ickr.com/photos/myxi/448253580
  7. Control
  8. The problem Given a dictionary of 1.000.000 search phrases, compare for each one the ?rst page of results on google.com and bing.com
  9. Don’t do it. It’s a violation of terms of service! (Sorry, I needed an example that required little or no explanation)
  10. Comparison
  11. The whole process
  12. Great! but... • Fetching, parsing and analyzing 2.000.000 pages of results will take a long time • we want to collect the statistics in a limited time and this sequential approach is not quick enough
  13. Distributed Computing Is a method of solving computational problem by dividing the problem into many tasks run simultaneously on many hardware or software systems (Wikipedia)
  14. Map Reduce "Map" step the master node takes the input, chops it up into smaller sub-problems, and distributes those to worker nodes. (Wikipedia)
  15. Problems: • How many nodes? • How many workers? • Distribution mechanism to feed the workers?
  16. What about queuing? • the master node takes the input, chops it up into smaller sub-problems, and publishes them in a queue • workers independently consume the content of the queue
  17. Here comes • RabbitMQ is an implementation of AMQP, the emerging standard for high performance enterprise messaging • It’s opensource • Written in Erlang
  18. Erlang? • general-purpose concurrent programming language designed by Ericsson • ?rst version written by J. Armstrong in 1986 • distributed • fault tolerant • soft real time • high availability
  19. Erlang - is coming back • Projects • CouchDB - RESTful document storage • WebMachine - REST toolkit • Nitrogen - web framework • Mochiweb - web framework • Ruby shops using it • 37 Signal - Camp?re • Heroku
  20. + Erlang It’s messages all the way down
  21. Install it • sudo apt-get install rabbitmq • sudo gem install tmm1-amqp Note: rabbitMQ must be v1.6.0 and amqp gem v 0.6.4 to follow the code in the slides
  22. Do it! - master node
  23. Do it! - worker node
  24. Get for free • Decoupling master/worker • Workers take care of feeding themselves • Flexible number of workers
  25. Behind the scenes Worker1 Master msg A Exchange Queue Worker2 Worker3
  26. Behind the scenes Worker1 Master Exchange Queue Worker2 msg A Worker3
  27. Behind the scenes Worker1 Master Exchange Queue msg A Worker2 Worker3
  28. Behind the scenes msg A Worker1 Master Exchange Queue Worker2 Worker3
  29. RabbitMQ • Multiple exchanges (and multiple types of exchange) • Multiple queues • Queues are connected by bindings to exchanges • Exchanges route messages to queues
  30. RabbitMQ • Exchanges and queues have names • BUT direct exchanges created implicitly are not public and don’t have name • Queues and messages are resident in RAM and can be persisted on disk (at a performance price)
  31. What and where Master (ruby) RabbitMQ (Erlang) Worker TCP/IP (ruby) Worker (ruby) Exchange Queue Worker (ruby)
  32. Problem #1 If a worker has a problem we might lose one or more messages http://www.?ickr.com/photos/danzen/2288625136/
  33. Solution - ACK in worker
  34. Acknoledgement • messages - 1 or more depending by prefetch settings - are passed to the client • the client processes and acknowledges the messages one by one • if the client connection closes => unacknowledged messages go back in the queue
  35. http://www.?ickr.com/photos/street?y_jz/2770586821/ Problem #2 No convenient way to control the workers
  36. System queue - worker
  37. System queue - control • save it as system_command.rb • ruby system_command.rb halt
  38. System queue Queue1 Worker1 Control script msg A Exchange Queue2 Worker2 Queue3 Worker3
  39. System queue Queue1 Worker1 Control script Exchange Queue2 Worker2 msg A Queue3 Worker3
  40. System queue msg A Queue1 Worker1 Control script Exchange msg A Queue2 Worker2 msg A Queue3 Worker3
  41. EventMachine
  42. EventMachine Is an implementation of Reactor Pattern • Non blocking IO and lightweight concurrency • eliminate the complexities of high- performance threaded network programming
  43. without EM with EM Ruby process Ruby process Time code code network network operation Free operation use network use network operation operation Callback result result Free Free
  44. EventMachine amqp gem is built on top of EventMachine => you’re in a context where you can leverage concurrent programming
  45. EM - Deferrables “The Deferrable pattern allows you to specify any number of Ruby code blocks that will be executed at some future time when the status of the Deferrable object changes “
  46. EM - Deferrables
  47. EM - Deferrables
  48. EM - Deferrables
  49. EM - Deferrables
  50. Deferrables without deferrables with deferrables GooglePage GooglePage BingPage BingPage Time
  51. http://www.?ickr.com/photos/philocrites/341850461/ Problem #3 How many of them? what are they doing?
  52. Heartbeat - worker
  53. Heartbeat monitor
  54. Heartbeat queue Worker1 msg A Worker2 Exchange Queue Monitor Worker3 msg B
  55. Heartbeat queue Worker1 msg A Worker2 Exchange Queue Monitor msg B Worker3
  56. Heartbeat queue Worker1 Worker2 Exchange Queue msg B msg A Monitor Worker3
  57. Clustering RabbitMQ - node A RabbitMQ - node B TCP/IP TCP/IP RabbitMQ - node C TCP/IP
  58. async vs sync Syncronous clients on github • famoseagle/carrot • celldee/bunny
  59. Rabbits on github • danielsdeleo/moqueue Tests/stubbing • Check out ezmobius/nanite “self assembling fabric of ruby daemons” • Keep an eye on tonyg/rabbithub implementation of pubsubhubbub (PubSub over REST) • auser/alice web app to monitor RabbitMQ
  60. More rabbits on github • tmm1/em-spec • eventmachine/eventmachine • tmm1/amqp • macournoyer/thin
  61. Q&A ?
  62. Thanks! (Thanks Mark!) Paolo Negri / hungryblank

Download
Distributed and concurrent programming with RabbitMQ and EventMachine Rails Underground 2009

 

 

Your download will begin in a moment.
If it doesn't, click here to try again.

Share Distributed and concurrent programming with RabbitMQ and EventMachine Rails Underground 2009 to:

Insert your wordpress URL:

example:

http://myblog.wordpress.com/
or
http://myblog.com/

Share Distributed and concurrent programming with RabbitMQ and EventMachine Rails Underground 2009 as:

From:

To:

Share Distributed and concurrent programming with RabbitMQ and EventMachine Rails Underground 2009.

Enter two words as shown below. If you cannot read the words, click the refresh icon.

loading

Share Distributed and concurrent programming with RabbitMQ and EventMachine Rails Underground 2009 as:

Copy html code above and paste to your web page.

loading
PDFCast Tags | Browse | DMCA Guidelines | Terms of Use | Privacy Notice | Contact | Facebook | Valid XHTML | Sitemap | Book Sitemap
© Copyright 2011 PDFCast.org

0.1039 - 4.41MB