lmorchard - Pebbling Club 🐧🪨

Autoscale based on metrics · Fly Docs

fly.io
2024-10-08T02:56:09.614Z
dev webdev flyio scaling hosting

Notes

The metrics-based autoscaler scales an app’s Machines based on any metric, such as pending work items or queue depth. Scaling on metrics other than requests or connections is useful for apps, like background workers, that aren’t running web services. Apps with services that need to scale based on HTTP requests can use the built-in Fly Proxy autostop/autostart feature for Machines.

Unfurl

{ "expires": 1728442563963, "status": 200, "bodyLength": 161652, "author": null, "date": null, "description": "Documentation and guides from the team at Fly.io.", "image": "https://og-images.fly.dev/image?template=light&text=Autoscale+based+on+metrics", "logo": "https://fly.io/static/images/favicon/apple-touch-icon.png", "publisher": "Fly", "title": "Autoscale based on metrics", "url": "https://fly.io/docs/launch/autoscale-by-metric/", "feed": null, "iframe": null, "lang": "en", "video": null, "audio": null, "duration": 481 }
Overview — Ray 2.34.0

docs.ray.io
2024-08-16T09:49:59.420Z
ai scaling python queues ml imported:raindrop

Unfurl

{ "image": "https://docs.ray.io/en/latest/_images/map-of-ray.svg", "logo": "https://docs.ray.io/en/latest/_static/favicon.ico", "title": "Overview — Ray 2.37.0", "url": "https://docs.ray.io/en/latest/ray-overview/index.html", "lang": "en", "failed": true, "failedAt": 1728192767394, "failedError": "TimeoutError: Promise timed out after 10000 milliseconds", "cached": true, "cachedAt": 1728192777307 }
Scaling GitHub

zachholman.com
2012-01-30T16:24:39.000Z
github scaling humans computers imported:pinboard

Notes

This type of scaling presents some interesting technical challenges. I'll dig into our development workflow and how we address concepts like scaling, deployment, code review, and testing.

Feed

Unfurl

{ "author": "Zach Holman", "description": "Written pieces, talks, and other bits by Zach Holman.", "image": "http://zachholman.com/images/front.jpg", "logo": "https://zachholman.com/apple-touch-icon.png", "publisher": "Zach Holman!", "title": "Scaling GitHub", "url": "http://zachholman.com/talk/scaling-github", "feed": "http://zachholman.com/atom.xml", "lang": "en", "failed": true, "failedAt": 1728194975497, "failedError": "TimeoutError: Promise timed out after 10000 milliseconds", "expires": 1728364898956, "status": 200, "bodyLength": 4251, "duration": 1429 }
Green Unicorn - Welcome

gunicorn.org
2010-03-31T17:41:20.000Z
python wsgi unicorn http webdev httpd performance scaling imported:pinboard

Notes

"Green Unicorn (gunicorn) is an HTTP/WSGI Server for UNIX designed to serve fast clients or sleepy applications."

Feed

Unfurl

{ "failed": true, "failedAt": 1728195487366, "failedError": "TimeoutError: Promise timed out after 10000 milliseconds", "expires": 1728365768554, "status": 200, "bodyLength": 7856, "image": "http://gunicorn.org/images/logo.jpg", "logo": "https://gunicorn.org/images/favicon.png", "publisher": "Python WSGI HTTP Server for UNIX", "title": "Gunicorn - Python WSGI HTTP Server for UNIX", "url": "http://gunicorn.org/", "feed": "https://github.com/benoitc/gunicorn/commits/master.atom", "lang": "en", "duration": 9917 }
Infrastructure for Modern Web Sites « random($foo)

randomfoo.net
2009-02-10T05:22:16.000Z
webdev infrastructure scaling imported:pinboard

Notes

"One of the things that I did when I wrapping up at Yahoo! was to begin to take a look at the current state of web frameworks. I ended up picking Django, but I have to say, I was disappointed with the state of what’s out there. Friends will have heard me bemoaning this sad state of affairs - that while Rails and Django might make CRUD easier, that the ORMs weren’t suitable for scaling beyond “toy” sizes, and that more importantly, they didn’t seem to address almost any of the pain points of building and maintaining a modern website."

Feed

Embed

Unfurl

{ "failed": true, "failedAt": 1728195527418, "failedError": "TimeoutError: Promise timed out after 10000 milliseconds", "expires": 1728365906810, "status": 200, "bodyLength": 35793, "author": "lhl", "date": "2009-02-04T16:53:36.000Z", "description": "One of the things that I did when I wrapping up at Yahoo! was to begin to take a look at the current state of web frameworks. I ended up picking Django, but I have to say, I was disappointed with the state of what’s out there. Friends will have heard me bemoaning this sad state of affairs - that w", "image": "https://randomfoo.net/junk/avatars/me", "logo": "https://randomfoo.net/favicon.ico", "publisher": "random($foo)", "title": "Infrastructure for Modern Web Sites", "url": "https://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites", "feed": "https://randomfoo.net/feed", "iframe": "<blockquote class=\"wp-embedded-content\" data-secret=\"6KnHgIhj11\"><a href=\"https://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites\">Infrastructure for Modern Web Sites</a></blockquote><iframe sandbox=\"allow-scripts\" security=\"restricted\" src=\"https://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites/embed#?secret=6KnHgIhj11\" width=\"600\" height=\"338\" title=\"“Infrastructure for Modern Web Sites” — random($foo)\" data-secret=\"6KnHgIhj11\" frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" scrolling=\"no\" class=\"wp-embedded-content\"></iframe><script type=\"text/javascript\">\n/* <![CDATA[ */\n/*! This file is auto-generated */\n!function(d,l){\"use strict\";l.querySelector&&d.addEventListener&&\"undefined\"!=typeof URL&&(d.wp=d.wp||{},d.wp.receiveEmbedMessage||(d.wp.receiveEmbedMessage=function(e){var t=e.data;if((t||t.secret||t.message||t.value)&&!/[^a-zA-Z0-9]/.test(t.secret)){for(var s,r,n,a=l.querySelectorAll('iframe[data-secret=\"'+t.secret+'\"]'),o=l.querySelectorAll('blockquote[data-secret=\"'+t.secret+'\"]'),c=new RegExp(\"^https?:$\",\"i\"),i=0;i<o.length;i++)o[i].style.display=\"none\";for(i=0;i<a.length;i++)s=a[i],e.source===s.contentWindow&&(s.removeAttribute(\"style\"),\"height\"===t.message?(1e3<(r=parseInt(t.value,10))?r=1e3:~~r<200&&(r=200),s.height=r):\"link\"===t.message&&(r=new URL(s.getAttribute(\"src\")),n=new URL(t.value),c.test(n.protocol))&&n.host===r.host&&l.activeElement===s&&(d.top.location.href=t.value))}},d.addEventListener(\"message\",d.wp.receiveEmbedMessage,!1),l.addEventListener(\"DOMContentLoaded\",function(){for(var e,t,s=l.querySelectorAll(\"iframe.wp-embedded-content\"),r=0;r<s.length;r++)(t=(e=s[r]).getAttribute(\"data-secret\"))||(t=Math.random().toString(36).substring(2,12),e.src+=\"#?secret=\"+t,e.setAttribute(\"data-secret\",t)),e.contentWindow.postMessage({message:\"ready\",secret:t},\"*\")},!1)))}(window,document);\n/* ]]> */\n</script>\n", "lang": "en", "duration": 3242 }
Joshua Schachter - Lessons Learned in Scaling and Building Social Systems | Kris Jordan

www.krisjordan.com
2009-01-23T18:43:17.000Z
delicious del.icio.us joshu yahoo webdev scaling design imported:pinboard

Unfurl

{ "failed": true, "failedAt": 1728195537391, "failedError": "TimeoutError: Promise timed out after 10000 milliseconds" }
Spock Proxy

spockproxy.sourceforge.net
2008-12-11T01:14:52.000Z
spock mysql sql scaling webdev database proxy imported:pinboard

Notes

"Spock Proxy supports range-based horizontal paritioning of a large MySQL database. The proxy intercepts SQL queries from the client, sends queries to the correct databases based on how the database is partitioned, then aggregates the results from each database and returns them to the client as a regular MySQL result set. "

Unfurl

{ "failed": true, "failedAt": 1728195537422, "failedError": "TimeoutError: Promise timed out after 10000 milliseconds", "expires": 1728365925550, "status": 200, "bodyLength": 26764, "image": "http://spockproxy.sourceforge.net/images/proxy_schema.png", "logo": "https://sourceforge.net/favicon.ico", "title": "Spock Proxy", "url": "http://spockproxy.sourceforge.net/", "duration": 6285 }
7 Stages of Scaling Web Applications - SlideShare

www.slideshare.net
2008-11-10T17:32:46.000Z
scaling webdev hosting optimization scalability architecture imported:pinboard

Notes

"Slides from LinuxWorld presentation by John Engates, CTO of Rackspace."

Embed

Unfurl

{ "failed": true, "failedAt": 1728195547405, "failedError": "TimeoutError: Promise timed out after 10000 milliseconds", "expires": 1728365933710, "status": 200, "bodyLength": 295476, "author": "David Mitzenmacher", "date": "2008-08-06T21:24:00.000Z", "description": "7 Stages of Scaling Web Applications - Download as a PDF or view online for free", "image": "https://cdn.slidesharecdn.com/ss_thumbnails/linuxworld-1218083370681251-9-thumbnail.jpg?width=640&height=640&fit=bounds", "logo": "https://public.slidesharecdn.com/_next/static/media/favicon.7bc3d920.ico", "publisher": "Slideshare", "title": "7 Stages of Scaling Web Applications", "url": "https://www.slideshare.net/slideshow/7-stages-of-scaling-web-applications/545001", "iframe": "<iframe src=\"https://www.slideshare.net/slideshow/embed_code/key/MyZp0QEWNmPluM\" width=\"427\" height=\"356\" frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" scrolling=\"no\" style=\"border:1px solid #CCC; border-width:1px; margin-bottom:5px; max-width: 100%;\" allowfullscreen> </iframe> <div style=\"margin-bottom:5px\"> <strong> <a href=\"https://www.slideshare.net/slideshow/7-stages-of-scaling-web-applications/545001\" title=\"7 Stages of Scaling Web Applications\" target=\"_blank\">7 Stages of Scaling Web Applications</a> </strong> from <strong><a href=\"https://www.slideshare.net/davemitz\" target=\"_blank\">David Mitzenmacher</a></strong> </div>\n", "lang": "en", "duration": 6819 }
Code: Flickr Developer Blog » Flickr Engineers Do It Offline

code.flickr.com
2008-09-28T02:47:30.000Z
flickr queues webdev scaling scalability imported:pinboard

Notes

"It seems that using queuing systems in web apps is the new hottness . While the basic idea itself certainly isn’t new, its application to modern, large, scalable sites seems to be. At the very least, it’s something that deserves talking about — so here’s how Flickr does it, to the tune of 11 million tasks a day."

Feed

Embed

Unfurl

{ "failed": true, "failedAt": 1728195547443, "failedError": "TimeoutError: Promise timed out after 10000 milliseconds", "expires": 1728365956549, "status": 200, "bodyLength": 39737, "author": "Kay Kremerskothen", "date": "2008-09-26T12:00:00.000Z", "image": "https://secure.gravatar.com/avatar/06e1b01eceade0a86c05b2a6ba1bb59e?s=68&d=identicon&r=g", "logo": "https://code.flickr.com/favicon.ico", "publisher": "code.flickr.com", "title": "Flickr Engineers Do It Offline | code.flickr.com", "url": "https://code.flickr.net/2008/09/26/flickr-engineers-do-it-offline/", "feed": "https://code.flickr.net/feed/", "iframe": "<blockquote class=\"wp-embedded-content\" data-secret=\"3aM2AxymrG\"><a href=\"https://code.flickr.net/2008/09/26/flickr-engineers-do-it-offline/\">Flickr Engineers Do It Offline</a></blockquote><iframe sandbox=\"allow-scripts\" security=\"restricted\" src=\"https://code.flickr.net/2008/09/26/flickr-engineers-do-it-offline/embed/#?secret=3aM2AxymrG\" width=\"600\" height=\"338\" title=\"“Flickr Engineers Do It Offline” — code.flickr.com\" data-secret=\"3aM2AxymrG\" frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" scrolling=\"no\" class=\"wp-embedded-content\"></iframe><script type=\"text/javascript\">\n/*! This file is auto-generated */\n!function(c,d){\"use strict\";var e=!1,o=!1;if(d.querySelector)if(c.addEventListener)e=!0;if(c.wp=c.wp||{},c.wp.receiveEmbedMessage);else if(c.wp.receiveEmbedMessage=function(e){var t=e.data;if(!t);else if(!(t.secret||t.message||t.value));else if(/[^a-zA-Z0-9]/.test(t.secret));else{for(var r,s,a,i=d.querySelectorAll('iframe[data-secret=\"'+t.secret+'\"]'),n=d.querySelectorAll('blockquote[data-secret=\"'+t.secret+'\"]'),o=new RegExp(\"^https?:$\",\"i\"),l=0;l<n.length;l++)n[l].style.display=\"none\";for(l=0;l<i.length;l++)if(r=i[l],e.source!==r.contentWindow);else{if(r.removeAttribute(\"style\"),\"height\"===t.message){if(1e3<(s=parseInt(t.value,10)))s=1e3;else if(~~s<200)s=200;r.height=s}if(\"link\"===t.message)if(s=d.createElement(\"a\"),a=d.createElement(\"a\"),s.href=r.getAttribute(\"src\"),a.href=t.value,!o.test(a.protocol));else if(a.host===s.host)if(d.activeElement===r)c.top.location.href=t.value}}},e)c.addEventListener(\"message\",c.wp.receiveEmbedMessage,!1),d.addEventListener(\"DOMContentLoaded\",t,!1),c.addEventListener(\"load\",t,!1);function t(){if(o);else{o=!0;for(var e,t,r,s=-1!==navigator.appVersion.indexOf(\"MSIE 10\"),a=!!navigator.userAgent.match(/Trident.*rv:11\\./),i=d.querySelectorAll(\"iframe.wp-embedded-content\"),n=0;n<i.length;n++){if(!(r=(t=i[n]).getAttribute(\"data-secret\")))r=Math.random().toString(36).substr(2,10),t.src+=\"#?secret=\"+r,t.setAttribute(\"data-secret\",r);if(s||a)(e=t.cloneNode(!0)).removeAttribute(\"security\"),t.parentNode.replaceChild(e,t);t.contentWindow.postMessage({message:\"ready\",secret:r},\"*\")}}}}(window,document);\n</script>\n", "lang": "en", "duration": 9709 }
I'm Going To Scale My Foot Up Your Ass - Ted Dziuba

teddziuba.com
2008-04-26T00:04:45.000Z
funny scaling scalability webdev imported:pinboard

Notes

"Shut up about scalability, no one is using your app anyway."

Unfurl

{ "failed": true, "failedAt": 1728195577487, "failedError": "TimeoutError: Promise timed out after 10000 milliseconds" }
random($foo): Internet Asshattery, Armchair Scaling Experts Edition

randomfoo.net
2008-04-25T16:23:01.000Z
webdev scaling socialsoftware imported:pinboard

Notes

"it really is true that if you haven't done it, that is: been intimately involved growing a social web app from prototype to large/giant-scale on a UNIX stack, then you really don't know shit."

Unfurl

{ "failed": true, "failedAt": 1728195577487, "failedError": "TimeoutError: Promise timed out after 10000 milliseconds" }
Working Notes on Consistent Hashing - Laughing Meme

laughingmeme.org
2008-03-19T22:44:23.000Z
scaling hashing consistenthashing webdev imported:pinboard

Notes

"Nice to see consistent hashing go from obscure to blindingly obvious in a few short whitepapers. "

Unfurl

{ "failed": true, "failedAt": 1728195587491, "failedError": "TimeoutError: Promise timed out after 10000 milliseconds" }
Tom White: "Disks have become tapes"

www.lexemetech.com
2008-03-19T22:41:58.000Z
disks tapes mapreduce scaling imported:pinboard

Notes

"In essence MapReduce works by repeatedly sorting and merging data that is streamed to and from disk at the transfer rate of the disk. Contrast this to accessing data from a relational database that operates at the seek rate of the disk"

Unfurl

{ "failed": true, "failedAt": 1728195587493, "failedError": "TimeoutError: Promise timed out after 10000 milliseconds" }
Programmer’s Toolbox Part 3: Consistent Hashing | Spiteful.com

www.spiteful.com
2008-03-19T04:33:11.000Z
webdev scaling hashing imported:pinboard

Notes

"Consistent hashing is a powerful idea for anyone building services that have to scale across a group of computers."

Unfurl

{ "failed": true, "failedAt": 1728195587493, "failedError": "TimeoutError: Promise timed out after 10000 milliseconds" }
Gojko Adzic » Two data streams for a happy website

gojko.net
2008-03-04T18:46:38.000Z
scaling webdev itoldyouso imported:pinboard

Notes

"One of the most important architectural decisions that must be done early on in a scalable web site project is splitting the data flow into two streams: one that is user specific and one that is generic."

Feed

Unfurl

{ "failed": true, "failedAt": 1728195587494, "failedError": "TimeoutError: Promise timed out after 10000 milliseconds", "expires": 1728366063130, "status": 200, "bodyLength": 27217, "author": "Gojko Adzic", "description": "One of the most important architectural decisions that must be done early on in a scalable web site project is splitting the data flow into two streams: one…", "image": "https://gojko.net/assets/gojko-adzic-heading.png", "logo": "https://gojko.net/assets/gojko-adzic-illustration-180.png", "publisher": "Gojko's blog", "title": "Two data streams for a happy website", "url": "https://gojko.net/2008/03/03/two-data-streams-for-a-happy-website/", "feed": "https://gojko.net/feed.xml", "lang": "en", "duration": 7120 }

0 - 50 of 15 items

10 25 50 100 250