Columnar storage is very effectively compressed so one "page" actually contains a lot of data (Parquet rowgroups default to 100k records IIRC). Writing usually means replacing the whole table once a day or appending a large block, not many small updates. And reading usually would be full scans with smart skipping based on predicate pushdown, not following indexes around.
So the same two million row table that in a traditional db would be scattered across many pages might be four files on S3, each with data for one month or whatnot.
But also in this space people are more tolerant of latency. The whole design is not "make operations over thousands of rows fast" but "make operations over billions of rows possible and not slow as a second priority".
Good points. I don't have a lot of experience with DuckDB in a production setting, but my team uses ClickHouse, where we ingest log and instrumentation data into materialized views at high volume. What I think saves the segmented/layered architecture there (ClickHouse calls them parts, but it's fundamentally the same thing) is that it's append-only, which means the "layers" don't go backwards, and a single row will never appear in more than one layer. But with a B+tree, the entire tree is mutable.
SAP is very good at what it is trying to do, which is to define, standardize, automate and run a business process, and it is equipped with a large library of premade processes so you don't have to reinvent the wheel.
It does not have good UX because good UX was never the objective.
That is not going to happen. Even if MAGA doesn't rig the midterms and the Democrats actually win something, they will just "reach across the aisle" and "work on healing our divided nation". Nobody will see any consequences for the suffering they caused.
it's not the consistent opinion that's the problem, it's the single issue (EU is bad) they are purportedly (i didn't check) focused on.
also, "EU is bad" is suspicious in itself because it can't possibly be that everything about the EU is bad. a good faith opinion will find some good things about the EU and be specific in what they are criticizing.
Do you have some links to how someone scaled up storage? I know that scaling up solar is easy, but I don't know of any nation that build significant storage.
> This is a common rebuttal, but not grounded in reality. Even assuming ~20% capacity factor for "apples to apples" comparison to legacy thermal and nuclear, solar and batteries are the cheapest form of power to install.
I looked it up because I was curious, according to Wikipedia average PV capacity factor is 25 % in USA, 10 % in the UK or Germany.
Nuclear has 88 % capacity factor worldwide. Meaning to replace 1 GW of nuclear installed capacity you need 8.8 GW of PV installed capacity in Germany or 3.5 GW of PV installed capacity in US.
Which might still be economically worth it, I don't know. But it is a number that surprised it.
It takes ~10 years to build a new nuclear generator from breaking ground to first kw to the grid, and tens of billions of dollars or euros. Germany deploys ~2GW/month of solar, the US ~4-5GW/month. Total global nuclear generation capacity is ~380GW as of this comment. At current global solar PV deployment rates, even assuming capacity factor delta between solar and nuclear, you could replace total global nuclear generation with ~18 months of solar PV deployment.
Yes, the biggest advantage of solar and wind is that they can be built as many small projects, instead of few gigaprojects we seem to have lost the ability to execute in the West.
Why is this even a crises? Sure there's fossil fuel price shocks but watching mission control for Artemis and comparing it to the Apollo missions the difference in tech can't be understated. We've made massive progress in only 50 years as a civilization collectively. We used to basically waste energy powering giant displays. Now we use a fraction of the energy on far better ones. 50 years from now we're likely to have so much solar and batteries deployed that it might actually hit "almost free" levels.
> It takes ~10 years to build a new nuclear generator from breaking ground to first kw to the grid
There is only one country on earth that can currently build a new nuke in 10 years. They are currently building more than the rest of the world combined.
For everyone else it’s 20 years at the absolute minimum.
Modern grids favour flexibility over fixed baseload generation (like nuclear) though. When you turn off a nuclear power plant its operating costs basically stay the same, which is horrible when you could cover your whole consumption with basically free solar/wind.
actually nuclear is terrible in a grid increasingly full of nearly-free variable sources (solar&wind). The nukes need to stay at 100% all the time selling their power at a high fixed price to have any remote chance of being economical. Cheap variables push nuke's expensive power off the grid during the day, and increasingly into the evenings with batteries. This is deadly to the economics of nuclear.
This is a brilliant example of DARVO: The US and Israel decided to attack Iran but somehow they had no choice and are actually the victims in all of this.
Columnar storage is very effectively compressed so one "page" actually contains a lot of data (Parquet rowgroups default to 100k records IIRC). Writing usually means replacing the whole table once a day or appending a large block, not many small updates. And reading usually would be full scans with smart skipping based on predicate pushdown, not following indexes around.
So the same two million row table that in a traditional db would be scattered across many pages might be four files on S3, each with data for one month or whatnot.
But also in this space people are more tolerant of latency. The whole design is not "make operations over thousands of rows fast" but "make operations over billions of rows possible and not slow as a second priority".
reply