In the context of the paper, the entire book seems to go downhill from the definition of ontology for me.
There is no benefit of using Gruber's ivory tower definition. A simpler explanation (e.g., it describes a structured framework that defines and categorizes the entities within a specific domain and the relationships among those entities) would have sufficed, and easier to digest.
Palantir is doing nothing revolutionary or "paradigm shift" when it comes to data and information organization. Their secret weapon is not introducing ontology to information.
Ching (1000BC?) classified reality into binary ontological primitives, created trigrams and hexagrams a combinatorial ontology. Aristotle introduced categories, substance, properties, relations, etc. Thomas Aquinas systemized Aristotelian categories into theological knowledge systems, and used structured classifications.
I am becoming curmudgeony as I see more and more of these reverse-research papers. Write the paper, then find references that fit the statement and use weasel words ...
unbelievable scene unfolds, deep-rooted disease of silos, paradigm shift, fatal flaws, forged in these extreme environments, eliminated to the absolute limit...
Palantir's secret weapon is the closeness and affinity to the DoD.
The tech stack ontological model is flexible like Salesforce so that it can be jammed into any task or contract quicky. It isn't engineered, it's glued in.
They're able to do this fast because they have a flexible model and because they have the friendly relationships.
Their moat deepens every year with every new integration.
It's smart as hell, actually. That's why they're swimming in money. And government contracts are about as lucrative as you can get.
Engineers turn their nose at this, but look who has tapped into this wealthy revenue stream. While we preen about good architecture, they can retire for a thousand thousand lifetimes.
> Palantir's secret weapon is the closeness and affinity to the DoD.
Is it a secret? I got an impression that it has been well known. How could you get any big number contracts without former secretaries or retired generals in your board or in your ‘consulting’ team?
Ontology is not a buzzword. It's precision nomenclature.
We've been using ontology well before RDF and the semantic web. It precisely describes their flexible engineering approach of using entities, definitions, and relationships.
If you read the comment a little more closely, it is very obvious that the "this" engineers turn their noses up at is the flexible model full of glue code, ala Salesforce, as opposed to "good architecture".
It's more or less in the same vein as pointing out that WordPress powered a massive chunk of the Internet despite violating almost every good coding practice you can name, and that getting things done is what makes money, not building ivory towers.
The fact that you turned that argument into some sort of anti American screed says much more about you than the parent.
To be fair, I had the same interpretation as OP here. One cannot have an earnest discussion of Palintir without at least implicitly including the privacy & military industrial complex associations of this company.
That is why I called it tone deaf, I admit the part about American Imperialism may have been unwarranted (may is in emphasis for a reason).
This engineer turned their nose at the bad architecture and glue code, but neglected to mention the total lack of morality from Palantir. I would argue that abandoning morality and aiding the American imperalist machine in its war against human rights and dignity, has been a much bigger reason for Palantir’s success then their lack of good engineering practice. They are willing to get paid for something most people morally object to. Lots of engineers are willing to abandon their craftsmanship if it pays well enough, few their morals.
Perhaps I read too much into this absence, in which case the post is only tone deaf, but I favor the read where this absence was intentional, in which case it is both tone deaf and American imperialist.
I don't think it's at all fair to make this kinds of inference from what was written, you'd have to make huge assumptions, and also take an ideological perspective as well. It might be a perfectly valid critique ... but it can't be at all inferred from the comment.
A lot of people, especially outside the US are going to look through a cynical geopolitical lens, which is not entirely unreasonable, so it's not 'surprising' at all that people would jump on this.
For example, I think Musk is a horrible person and I view all of his statements through the 'lens' of the fact he is lying, confabulating and he's a jerk.
But - I mean, SpaceX does work, it's by all means a pretty good company (work-life balance not withstanding).
That admiration is the tone deafness I perceived. It comes across as “we gotta hand it to ISIS” in its best interpretation.
Palantir has been on Amnesty International’s list of companies aiding in human rights violations since 2020, in particular for aiding DHS and ICE in illegal deportations and family separations, in 2023 the company provided tech to the IDF which was then used in the Gaza Genocide, the company prided it self of it (and consequently a lot of their staff resigned as their morality did not allow them to work there). This is just to say we are not just talking about government spying here, Palantir is a major participant in many of the worlds worst human rights abuses of the past decade.
Palantir is probably the company on the planet right now who is perceived by the general public as the most evil, and I for one think this company deserves this reputation. One does not, in fact, got to hand it to Palantir.
Perhaps gov contractor speak would be more accurate. I'd think a corp with an org sufficiently aligned to their business value and profit motive wouldn't stand for the fancy speak either
Pretty much any SQL book will cover it those, and there's a bunch of online SQL tutorials. UDF means user defined function, so if there's some function you want to perform in SQL but that function doesn't come out of the box, you can just write your own. And those can be defined in non-SQL syntax, such as UDF's written in python or C++, which can be pretty handy.
I'm having a blast with using PostgreSQL to hold unstructured data in JSON-B and then creating views off of that that "feel" like more conventional SQL.
If performance becomes an issue, just turn it into an MV... and then consider some indexing on the JSON itself.
The paypal mafia are all about stories. They can attract talent and investor money with those stories, but they are just tall tales full of hype, and people are catching on (ok that last bit might be a hopium).
I find myself distinctly unimpressed by the idea that slapping a nice UI and some TS/SCI controls on top of a graph database — the latter being something that NSA did, with considerably more sophistication, years prior in a Neo4J fork — is some kind of brilliant conceptual moat. Graph DBs are useful for certain kinds of problems, which happen to map well to counterterror social mapping strategies, this is nothing particularly new or noteworthy.
I've learned this stuff as a hobby, so take it with a grain of salt. I'm not a specialist.
OWL 1, for example, has stuff like transitive properties (the classical example is A ancestorOf B, B ancestorOf C, therefore I can infer A ancestorOf C if I annotate ancestorOf as a transitive property).
Union, equivalence, inversion, symmetries, cardinality. Those are all possible to represent symbolic in OWL ontologies.
They're also neatly separated in different types (OWL Lite, OWL DL, OWL Full). OWL Lite and DL for example are proven to be decidable (you won't get some halt when doing inference, no matter what).
I know there are plenty of database engines to store triples and graphs, and plenty of reasoners out there.
I haven't studied OWL 2 yet or newer stuff like SHACL, but I know it's supposed to be even better.
Palantir is just "Cambridge Analytica" redux but with more money/connections/data/breadth/depth/etc. Their ethical/moral stance i will leave it to you to infer.
palantir doesn't do revolutionary things in terms of back-ends. matter of fact, their apps are at best mid. I'd rate them 3/10 compared to alternatives that can do similar things. Their front end is the real differentiator.
Their bread-and-butter is a few things
1) Willing to do dirty/harmful things no one else will touch
2) Making data and data analysis accessible to cops, dhs, anyone that is especially tech-averse (many police departments disqualify based on IQ test results measuring too high). You can type in a license plate, a name, an address, scan a face and it will show you every relevant information, but also contextualizes it and enriches it with any other data. You could to this in excel, postgresql, bigquery, etc.. but palantir gives these people simple text boxes, buttons, and links.
3) Their forward deployed engineers are great at what they do. They station their guys wherever Palantir is being used, and they'll work very closely to get things done. to make sure all problems are solved asap, and its users are very well educated on the usage of the platform.
This post looks like it's written by AI, but assuming it is in earnest, it isn't really ontology, at least no more than object oriented programming is ontology. Excel is all about numbers, palantir is all about people (or people-documents). It is simpler than excel and has BigQuery level analytical power behind it, and the human touch to make that interaction go over really well.
I said it's mid because you could do a lot more with just the dataset and queries. You could even possibly do more with command line tools and hoards of data files (minus the OCR and document scanning they do, as well as LLM/NLP). but that isn't accessible and takes a lot more time. Not to mention normalizing, extracting and structuring wildy unstructured data isn't easy. But with BigQ for example, it is done plenty, you just hire a team to do that for you typically.
Their ecosystem is basically google search (including image, reverse image,video,etc..) but much more targeted and oriented towards displaying collated data from hoards of structured and unstructured data (including pdfs, docx,etc..). I would prefer grep, bigquery,splunk myself. but for end users, palantir is unmatched in my experience.
But I'm not selling them here, I'm trying to communicate the power at the disposal of those who use palantir's platforms. Google could have crushed them any time, except even for Google the type of work required was too ghoulish and reputationally risky.
Even with MS copilot(lol), chatgpt, gemini,etc.. running as agents, they're not as simply as palantir's stuff is for searching your data. and you don't have specialists integrating all your data onsite either.
Ultimately, the bigger problem is that even in crowds like HN's, no one seems to have a good idea of what should be done about governments abusing datascience so efficiently. Every answer comes back to red-tapes and regulations, possibly criminal consequence. Are you willing to give up the liberties tech has enjoyed so that future generations can be well, and have shot at peace and prosperity? (ours is too far gone in my opinion)?
China is doing this too, but much more efficiently, much better and at a greater scale. but their society has accepted this, and traded certain liberties for social stability and economic prosperity. The west hasn't done that. lawmakers and the public at large need to be informed by those in tech about these things so informed decisions could be made.
> Ultimately, the bigger problem is that even in crowds like HN's, no one seems to have a good idea of what should be done about governments abusing datascience so efficiently.
The general-populace/crowd/mob has already lost this game. Govts/Companies (all of them irrespective of ethics/democracy/etc.) are doing what they want with data and datascience. The populace is easily propagandized/distracted from reality and can be easily cocooned.
The only recourse left for the individual is to learn and start playing the game himself. Fortunately the new tools are a great help in this asymmetric warfare. Organizations like EFF/OpenSource/GNU/etc. need to take the lead on this since most people are like sheep when it comes to uses/misuses of technology.
Palantir is just "Cambridge Analytica" redux but with more money/connections/data/breadth/depth/etc. Watch this old presentation by their then ceo Alexander Nix and extrapolate to today's AI world - https://www.youtube.com/watch?v=n8Dd5aVXLCc
Finally go and read the works of George Orwell, Edward Bernays, Jacques Ellul, Marshall Mcluhan, Noam Chomsky etc. on the whole subject of Propaganda/Manipulation to really understand where we are now.
You can't fight this with tech. The problem is not technological. You don't have the data to fight with either. You could live in an igloo in the arctic with no tech on hand you'll still show up in palantir's hits (unless you were born there and have nothing to trace you back to civilization). Those organizations you mention might help with corporate surveillance capitalism, but not government data mining and surveillance. Palantir is not the same as Cambridge analytica, they're more similar to Microsoft and Google. In simpler terms, they provide the sql db, sql client, and the db admins to manage the data, as well as support to train users. They don't do the analysis and action on objectives for governments, they're not a consultancy firm like cambridge.
Under rule of law, you need laws. Outside of rule of law, well..
The idea is to protect oneself by using various techniques of deception/reflexive control through stratagems like "poisoning the well", "borrow a knife", "misdirect/misinform" etc. to present different contradictory information/personas/overload-data to "The System" thus obfuscating/masking reality.
You mess-up the raw data and then use reflexive control techniques to allow them to draw inferences from a set that you have predetermined.
Technology, mass commoditized and given easy access to, is central here and should be spearheaded by the organizations i mentioned due to their reputation cachet eg; Tor/Tails/Proxies/Vpns/Secure OSes/Protocols/etc. Simultaneously they should also push for legislation for stricter govt. control over data usage/companies eg. GDPR/etc.
The populace also needs to educate themselves on techniques of Propaganda/Manipulation by reading the works of the authors i mentioned earlier. This will sharpen their critical thinking skills which is a must-have today.
Palantir is very much in the vein of Cambridge Analytica except that they have expanded vastly on its theme and gained "legitimacy". People need to study all about CA and then extrapolate it to today's AI capabilities if they want to understand the shape of things to come.
Some References:
Deception: The Invisible War Between the KGB and the CIA by Edward Jay Epstein; For history/ideas/themes/motivation for further study on a complex subject.
You still don't get what palantir does. they're not necessarily mining online activities of people. although they do that almost certainly. but it's things like where you life, your movements, your associations, work place, etc.. that may seem harmless but they go layers and layers deeper to presume things. For example if someone you know has a friend who is a gang member, they might associate you without your knowledge with that person. the physical stuff is hard to beat by anyone. lots of cameras, radio sensors (wifi/bt), payment transactions, ISP data,etc..
Even if they worked like you think they do, simply downloading tails, or using a VPN alone will harm you in person much more than you think. You might be driving in peace, a cop would put in your license plate and you'd be flagged from heavy vpn usage. if they can't get enough additional information on you to dismiss you as harmless, that will do you more harm than good. No facebook,reddit,etc.. then that means you're up to no good. There is not tech solution for this, it is a human problem, the problem is people.
There is an old adage in hacking circles "physical compromise is total compromise", these people have physical power and control over you. Ever heard of the $5 wrench method of defeating cryptography?
Like you, many are wasting efforts in trying to find technical solutions, instead of communicating the impact of how tech is being abused to decision makers and the public.
are you disputing the "many" part because the lawsuits are limited in number? that aside, everything in that search results supports my statement which you quoted. they won't tell applications "you're too smart", the ones that do get sued, and those are the ones you hear about. police hiring is much more insidious than that, it's more akin to gang member recruitment. But, I will say that regional variations exist. Smaller towns or well reformed major cities might not have these issues. But for example the LA sheriffs department is not only a plain-and-simple gang, you literally have to have gang associations and participate in race-driven hostility against specific groups to be accepted.
This hiring practice is informal, similar to how last names influence hiring. the claim is also that they get rid of or reject applicants that are too smart. that doesn't mean they require applicants to be stupid, that's why as your search shows, the average IQ for cops is close to the general public's median.
Compare that with lawyers. One can argue that he people that are licensed to use violence to enforce the law should have the same level of cognitive capacity and store of legal knowledge as those who litigate the law in court. I hope you agree with that.
The claim that this has happened in the past is obviously well-supported. The claim that it's broad practice or representative in any general way of policing is more or less completely unfounded.
In fact, general cognitive tests of any sort aren't common recruiting practice. States have standardized reading comprehension and writing tests, like CA PELLET-B, but there's no high-end cutoff score for them, and there's also no direct analog between them and IQ.
I think we can all agree that police departments are allowed to reject applicants based on IQ. Cognitive tests are common from what googling I've done, although only about 10 states have IQ tests specifically. It's also hard to prove that candidates get rejected based on their IQ, it's not like this would be a department policy. Whether they exercise their right to do so, or how many exercise that right, there is no hard data that I could find, and you might be right that this is a bit exaggerated.
However, from direct experience, and the endless videos of police interacting with citizens out there, it's hard to not to draw conclusions.
I would imagine applicants would be told they're not a culture fit or something, similar to any other workplace that uses informal reasons to reject applicants. I would even agree that if you pass the right vibe-check, having high-IQ might not be an issue (e.g.: white supremacist tendency, or gang membership).
Google AI overview regarding cognitive testing (sorry, too lazy to dig up specific sources, i assume you can google the same things I did and find out):
While specific, standardized, state-wide cognitive test mandates for police hiring are not uniform across the U.S.,
approximately 37 states mandate or conditionally require psychological screening, which often includes cognitive assessment components. These evaluations are typically part of a broader, mandatory, and highly competitive selection process that often includes reading, writing, and, in some cases, critical thinking, or scenario-based assessments like the FrontLine Testing System.
I'm not sure how interesting you can make an axiomatic derivation of the claim that police departments widely have upper-end IQ cutoffs (or, more likely given that virtually no police departments administer actual IQ tests, high-end cutoffs on their existing domain-specific written tests).
The fact is that there are a very small number of cases where tiny departments decades ago (a) administered IQ tests for recruits, and (b) disqualified at least one candidate for exceeding a threshold. That's all the empirical evidence there is. That evidence cannot itself support the weight of the extraordinary claim that police departments generally disqualify applicants for exceptional cognitive skills.
In fact, the opposite thing is more likely to be the truth.
> In fact, the opposite thing is more likely to be the truth.
If you can't prove the former, you can't prove that either. It is speculation, but not unfounded speculation. a small number of cases were taken to court, and since those cases were found in favor of police departments, no further cases we taken to court. Police departments don't record shady practices like this either.
There is no definitive public data on which departments administer IQ tests, your supposition that it is a small number is unfounded. Nearly all administer cognitive tests, psychological and/or psychometric tests under various names are almost always administered, and from what I was able to look up on the subject, they usually contain IQ tests. In a corporate setting, I've taken a pre-hire psychometric test myself, it was in every way the same as an IQ test but they didn't use that term.
We are left with no choice but to speculate, the historical cases you mentioned laid the ground work. reason dictates that ground work is followed by others. as i mentioned, if inference is all that is left, we're left with public documentations of police conduct and hear-say, and all of that supports the original claim that this is a widespread practice.
You're asking us to believe the extraordinary claim that police departments view high intelligence as a liability and not an asset, and the evidence you're presenting barely exists at all, and the countervailing evidence is significant. That's all.
You haven't presented a single "countervailing evidence", or even a claim of one. You can't sue a police department if there is precedent that ruled they have the right to do that thing, so what are you expecting other than lawsuits? I mentioned already that they don't document these things. Lawsuits settled the matter, and I won't repeat for the 3rd time all the other corroborating evidence I mentioned.
I agree that evidence in general is significant, countervailing or not. I already changed my mind and conceded based on evidence that a lot of this is speculative, and exaggerations might exist. I'm more than open and receptive to evidence to support that high-iq or doing well on cognitive tests does not affect hiring in a negative way at all. but if empirical evidence to that effect existed, it would have settled our debate much sooner.
Let me use an analogy. Let's say the courts ruled 20 years ago that tech companies can refuse to hire employees that score really well on IQ tests for some reason. Over that time, the quality of work in the tech industry by developers declines rapidly. however, companies have no need to formally use that as a reason for not hiring due to PR reasons, and candidates can't sue them due to precedent. The most reasonable deduction is that a significant number of tech companies have followed suit and using high-cognition to reject candidates. outliers are to be expected. This is further corroborated by the fact that like police departments corporations copy-cat trends a lot (e.g.: RTO, layoffs,etc.. with companies. with PD's: militarization, treating the public as subjects, using "I feared for my life" to excuse behavior that would have had them hung not too long ago). Trends, behaviors, precedent and witness testimony. People are tried for murder for far less.
OK. You seem pretty confident. Let's make it easy on you. Name a single police department in the United States from which you can present evidence from the past 10 years that they have an IQ or general cognitive cap on applicants.
Should be straightforward, since the root of this thread confidently asserts that police generally disqualify applicants based on high IQ.
It doesnt matter who does search well, if you understand the implications of the theory of bounded rationality and what claude shannons information theory tells us about bandwith limitations of individual and group cognition.
What is this AI slop doing at the top of HN? Come on, you don't even have to click through to know it's slop! It even has an en dash right in the title!
#### The Paradigm Shift Brought by Palantir: Ontology as an Operational Layer
The *"Ontology"* strategy by Palantir, explained in this book, is a paradigm shift that fundamentally breaks this deep-rooted disease of silos.
In the context of knowledge engineering and the semantic web, the widely cited academic definition of "ontology" is an "explicit specification of a conceptualization" by Gruber (1993).
Furthermore, Studer et al. (1998) expanded on this, proposing the definition of a "formal, explicit specification of a shared conceptualization."
This transition from "data just for viewing" to "data that directly drives the business" is the key to true digital transformation in the AI era.
[end]
Just gave me brain damage. Please for the love of god just go straight to the point. Just give me the prompts that wrote all of this.
I worked at a company where the VPs etc spewed the word "ontology" all the time. We were never sure if they knew they were spewing bullshit or they really believed what they were saying was real.
Ontology is one of those fancy words that sounds important but is basically, as another poster pointed out, a standardized vocabulary.
It’s just a consulting firm with connections to the Trump administration through Thiel. That’s why they need forward engineers. It’s not a real platform. Ontology is as far as I can tell, a marketing buzzword, mostly repeated by bots to pump Palantir.
Eccentric CEO (we already have Elon another is one too many …) wildly overpriced stock and company and also one without any ethics that’s super close to the Trump admin … I for one am praying this company collapses. I’d love to read a HN post about how they go chapter 11 and leave the public eye.
In the context of the paper, the entire book seems to go downhill from the definition of ontology for me.
There is no benefit of using Gruber's ivory tower definition. A simpler explanation (e.g., it describes a structured framework that defines and categorizes the entities within a specific domain and the relationships among those entities) would have sufficed, and easier to digest.
Palantir is doing nothing revolutionary or "paradigm shift" when it comes to data and information organization. Their secret weapon is not introducing ontology to information.
Ching (1000BC?) classified reality into binary ontological primitives, created trigrams and hexagrams a combinatorial ontology. Aristotle introduced categories, substance, properties, relations, etc. Thomas Aquinas systemized Aristotelian categories into theological knowledge systems, and used structured classifications.
I am becoming curmudgeony as I see more and more of these reverse-research papers. Write the paper, then find references that fit the statement and use weasel words ...
unbelievable scene unfolds, deep-rooted disease of silos, paradigm shift, fatal flaws, forged in these extreme environments, eliminated to the absolute limit...
Gag me.
reply