True, but different networks have different constraints and require different solutions. Data warehousing, which is likely what most of GS's data is doing, is very different than trying to scaling a near-real time data access system.
Payment processing only deals with one type of data: money. This gives you as many shortcuts as the number of constraints it imposes.
Airline control and departure systems are probably as close to a typical modern web app as you'd get from your list.
My point is that while there certainly are engineers that have worked with high scalability issues without ever touching the web, they have also likely been solving slightly different problems.
P.S.: Other systems that require high availability but are not the web: telephony and cell communications, broadcasting and doomsday devices.
According to this link (after a few seconds of googling), Visa's network was down for 8 minutes in the five years ending 2001. Do you have citations for any of your claims?
The Visa figure I heard at the Computer History Museum years ago, it was first decades so that may have changed.
Goldman Sachs told me when I was writing a proposal for them years ago that they have over a petabyte of data stored. The web is 80-200TB depending on who you ask. A single department there responsible for the program based trading would alone have an entire copy of the web (and parts of the deep web, and all of twitter, etc.) since they construct those whack trading apps that suck everything up and analyze it for signals. If there are any quants on here they could tell you about this more.
One of my professors told me YouTube is hosting approximately 100TB of new videos every month, I think 200 TB is off by very, very much. If 200TB was the case I could afford to buy disks to store the entire web with my yearly salary...
They don't have to add features quickly because they spend years designing and building them based on a spec that somebody else has spent years designing all so that when it launches, you can go online and book an airline ticket or make a transaction or whatever other essential daily activity and not have to think about what is taking place, let alone see an error screen or 404
no what I am saying is that it is unusual for them to get themselves in a situation where they have to suddenly code out of a hole at 'startup pace' (whatever that means)
startups don't have a monopoly on working hard or working fast, and it is arrogant to generalize about both .NET and enterprise developers in that way since we are all in one way or anther standing on the shoulders of earlier enterprise work (where do you think what we call 'nosql' and think is new and grovvy, was first used?)
Totally fair. First, apologies if it sounded like I was suggesting startups have a monopoly on working hard and fast. Of course I never meant anything like that at all.
Second, arrogant really isn't fair because I certainly never gave an assessment of my own abilities or value.
Third, totally.. .NET stands on the shoulders of the same stuff the infrastructure that runs most of the internet stands on. It's just that .NET doesn't run most (or even lots and lots of) the internet.. so to suggest that the .NET development community is less likely on average to be ready to build Facebook doesn't seem like blasphemy.
That's very different from suggesting the .NET camp isn't full of awesome, hard-working developers... but really, I apologize if it comes off that way. Definitely don't mean to suggest it in the least.
There's a flaw in your logic which makes it seem like blasphemy. It's not enough to say that there aren't many .NET examples; you have to show that the proportion of .NET people who do good .NET work is lower than the proportion of PHP/etc. people who do good PHP/etc. work. With PHP in particular, I would guess that it's easily true that a higher proportion in .NET would be better equipped.
Not good work. We're not talking about good work. We're talking about fast work that runs on the web, works at scale, and allows the organization to pivot easily. You certainly won't catch me suggesting PHP is some sort of awesome language.. it's not. But it's language that lives on the web, and if I was building something on the web and had my pick of a random .NET developer and a random PHP one, I'd take the PHP one... because chances are the .NET developer has never deployed something on the internet, and chances are the PHP dev has.
It is a fallacy to think that the web is the only large network that requires scale.
Goldman Sachs alone stores more data than the entire web
The Visa network has had 4 seconds of downtime in decades
Most airline systems took decades to engineer