{"id":2000,"date":"2012-07-13T12:29:37","date_gmt":"2012-07-13T11:29:37","guid":{"rendered":"http:\/\/t-machine.org\/?p=2000"},"modified":"2012-07-13T12:30:26","modified_gmt":"2012-07-13T11:30:26","slug":"dropbox-tech-scaling-some-great-some-not-so-great","status":"publish","type":"post","link":"http:\/\/new.t-machine.org\/index.php\/2012\/07\/13\/dropbox-tech-scaling-some-great-some-not-so-great\/","title":{"rendered":"Dropbox tech scaling &#8211; some great, some not-so-great"},"content":{"rendered":"<p><a href=\"http:\/\/eranki.tumblr.com\/post\/27076431887\/scaling-lessons-learned-at-dropbox-part-1\">&#8220;I was in charge of scaling Dropbox &#8230; from roughly 4,000 to 40,000,000 users. &#8230; Here are some suggestions on scaling&#8221;<\/a><\/p>\n<p>The first section is a WTF &#8211; the guy advocates deliberately over-taxing your servers, without a good explanation. I&#8217;ve got some guesses at why they did it &#8211; but in the general case I&#8217;d say: never do this. Only do it when it&#8217;s obvious, because you have a specific reason to do so (and you really know what you&#8217;re doing).<\/p>\n<p>The rest is a lot clearer, good advice there.<\/p>\n<p>Also, IMHO worth reading for this part alone:<\/p>\n<blockquote><p>\n&#8220;Let\u2019s say you\u2019re trying to debug something in your webserver, and you want to know if maybe there\u2019s been a spike of activity recently and all you have are the logs. Having graphing for that webserver would be great, <\/p>\n<p>[ADAM: but &#8230; often you don&#8217;t have the right set of graphs set up, and it takes a while to do that &#8211; no use if the server is in trouble *right now*]<\/p>\n<p>Apr 8 2012 14:33:59 POST &#8230;<br \/>\nApr 8 2012 14:34:00 GET &#8230;<br \/>\nApr 8 2012 14:34:00 GET &#8230;<br \/>\nApr 8 2012 14:34:01 POST &#8230;<\/p>\n<p>You could use your shell like this:<\/p>\n<p>cut -d\u2019 \u2019 -f1-4 log.txt | xargs -L1 -I_ date +%s -d_ | uniq -c | (echo \u201cplot \u2018-\u2019 using 2:1 with lines\u201d; cat) | gnuplot<\/p>\n<p>Boom! Very quickly you have a nice graph of what\u2019s going on, and you can tailor it easily (select only one URL, certain time frames, change to a histogram, etc.).&#8221;\n<\/p><\/blockquote>\n","protected":false},"excerpt":{"rendered":"<p>&#8220;I was in charge of scaling Dropbox &#8230; from roughly 4,000 to 40,000,000 users. &#8230; Here are some suggestions on scaling&#8221; The first section is a WTF &#8211; the guy advocates deliberately over-taxing your servers, without a good explanation. I&#8217;ve got some guesses at why they did it &#8211; but in the general case I&#8217;d [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[33],"tags":[],"_links":{"self":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/posts\/2000"}],"collection":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/comments?post=2000"}],"version-history":[{"count":0,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/posts\/2000\/revisions"}],"wp:attachment":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/media?parent=2000"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/categories?post=2000"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/tags?post=2000"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}