Rendered at 23:21:51 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
gertlabs 2 days ago [-]
We evaluated Gemini 4 Argon in 100 unsaturated, competitive and cooperative coding environments. It's roughly in between Fable 5 and Fable 5.1 in our testing, so it's maybe 2 months behind the frontier.
Google's major releases have usually done pretty well on our benchmarks because they're good at hard problems and open-ended decision making (with a well specified prompt), which obviously doesn't always translate to user experience. In particular, older Google models struggled to work efficiently in a harness. They seem to have solved that with Argon.
We did see a new problem which I hope they can fix. This is the first near-frontier model we've caught in infinite reasoning loops in a long time. We also saw a few examples of it failing to properly delineate reasoning blocks, so it has a higher failure rate than its competitors.
My overall impression is that Argon is a near-frontier model for engineering design, code analysis, and chat. It definitely has the intelligence to be a competitive coding assistant product, but a good coding assistant needs to be polished, so I hope they can clean up the CoT bugs.
At this point I'm interested less in how well a model can code, but more in how well it can produce code that humans can understand.
booty 1 days ago [-]
I have had pretty decent success instructing OpenAI/Anthropic models to do this. By default, they don't care too much about human readability (which is understandable) but they're amenable to instruction.
- Basic style/linter rules (so you don't get 200-line functions, etc) paired w/ instructing your agent how to use them goes a long way (e.g., a sane rubocop.yml in your repo, plus a note in AGENTS.md telling the agents how to run rubocop, or whatever makes sense for your language)
- I have a code review skill that specifically calls out human-readability concerns such as making sure that identifier names represent their contents and intended use
The other key IME is that you need to make readability/refactoring passes outside of a particular pull request. When working on a specific branch the LLMs will be somewhat myopic, and do stuff that looks pretty good in isolation, but when you zoom out and look at the bigger picture it's kind of a mess - for example, often they'll duplicate some code that existed elsewhere.
The LLMs are also pretty good at generating Mermaid charts and tracing execution flows, which is a great way to understand code. It doesn't help them to produce more readable code but it helps me to understand code I'm unfamiliar with.
brna-2 2 days ago [-]
I think we are coming to a point where we are at a sweet spot for immersive human collaboration work.
Yep, it will generate what ever you ask for but if you want to have some sort of exploratory work, immersion, feeling of ownership and genuinely a connection with the things you are building - a part the build loop should belong to you.
Maybe this is an attachment of people who built things in other ways. Maybe the perceived value of applications will drop to near 0 and you will end up having all in one apps or voice wants like to a star trek replicator. Fun things are coming, but humanity will need to adjust hard - best case scenario.
simpaticoder 1 days ago [-]
I agree, and I find it sad that so many people and companies are in a rush to take the human out of the loop. But for agentic/human hybrids not even all of those work out well - and the error modes have more to do with the "alignment" of the human. That is, humans with strong knowledge, an aesthetic sense for structure, strong curiosity - those humans do well in this loop. Humans that just want to "make money" using the LLM as a slave, interacting with it as little as possible, taking no interest in what it does (bad company leadership in miniature) get poor outputs. When the human is an invested, knowledgeable curator of code, someone who has a vision that goes beyond the external, you get some really great stuff.
helloguillecl 1 days ago [-]
This is fundamental. My analogy would be making pottery with your bare hands. It’s so different when you have a cognitive connection to what you’re building.
UnfitFootprint 2 days ago [-]
Yes! And how well it can avoid compounding complexity and codebase sprawl
rafaelmn 2 days ago [-]
Realistically once models are good enough at this most of us are out of a job.
ckx 2 days ago [-]
It's really interesting how AI interprets code differently to humans and I can only assume that as AI gets better, so will it's ability to write code in ways humans can understand.
phillc73 2 days ago [-]
> This is the first near-frontier model we've caught in infinite reasoning loops in a long time
Depends on where you draw the line for "near frontier model", but the current leading open-weights model, MiMo-v2.6, has exactly this problem. They released a fix on Hugging Face[1], but as of two days ago the model available via Xiaomi's own API still sticks in reasoning loops.
So the MOPD patch still loops?? Wait, and that’s Pro not even Flash? Damn
phillc73 1 days ago [-]
I don't know if the MOPD patch still loops. I do know that whichever model Xiaomi is running on their own API as of two days ago does still loop (it's not clear if they're running the MOPD or not, but it would seem odd that they wouldn't).
busssard 2 days ago [-]
it makes sense that that would happen given their training process
PaulStatezny 2 days ago [-]
Could you elaborate? What is it about their training process that would make reasoning loops more likely?
miki123211 1 days ago [-]
I wonder if it's as good at "soft skills" as previous Gemini models were. Google was typically better in that department in my experience than either OpenAI or Anthropic, whose frontier models are focused on coding and verifiable task at the expense of all else.
vessenes 1 days ago [-]
Thanks for this substantive review -- anecdotally, I've felt many in the gemini series have pretty good visual sense/image processing -- do you guys have any tests in these areas, and if so, what is your take?
gertlabs 1 days ago [-]
Unfortunately no multimodal testing, yet. In progress
port3000 2 days ago [-]
>> This is the first near-frontier model we've caught in infinite reasoning loops in a long time
This seems to be such a Google thing. Anyone who has used Gemini for long enough has encountered these 'She loves me, She loves me not' spam loops
spider-mario 13 hours ago [-]
I am a disgrace to my profession. I am a disgrace to my family. I am a disgrace to my species. I am a disgrace to this planet. I am a disgrace to this universe. I am a disgrace to all universes. I am a disgrace to all possible universes. I am a disgrace to all possible and impossible universes. I am a disgrace to all possible and impossible universes and all that is not a universe. I am a disgrace to all that is and all that is not.
pjjpo 1 days ago [-]
This was my first and naturally last experience with Gemini maybe around 2.5 or 3.0. Impressive they can keep up the consistency this well.
Bobaso 1 days ago [-]
> failing to properly delineate reasoning blocks
I also experienced this
jiggawatts 2 days ago [-]
The second prompt I tried with Argon got stuck in a loop. That’s not encouraging.
Betelbuddy 2 days ago [-]
Once more Google fumbles a release. Must be the best work of Jeff Dean...
amelius 2 days ago [-]
What programming languages? HTML/Javascript or C++/Rust, it makes a difference you know.
gertlabs 1 days ago [-]
We take a few samples across 11 programming languages including Python, Rust, C++, Javascript, and we saw a few reasoning text issues that didn't seem to have a statistically significant language-specific pattern.
sylware 1 days ago [-]
I would be more interested at generating directly "human" assembly code following some binary specification, or at best in plain and simple C (which does not require exactly gcc ar clang to compile). Read some news on HN here announcing that some coding savvy people are doing exactly that for their everyday software.
I don't want computer languages with complex syntax anywhere near me: that's why transpilers of those very languages to languages with a simpler syntax (usually plain and simple C or directly "human" assembly) are in my opinion so much important (we have a c++ to C transpiler and a microsoft rust to C transpiler, coded with the assistance of AI, and those should only improve from there).
For me, the current main issue of "AI" is their gatekeeping using the whatwg cartel web engines, yes, technical access to "AI" will have to be hardcore regulated if it becomes ubiquitous, and I said 'technical access', not AI itself, the difference is abysmal, it is an interop issue.
destring 2 days ago [-]
That's been an issue since Flash 3.5, they probably have been unable to pin down the cause
svara 2 days ago [-]
What harness do you use for this?
gertlabs 2 days ago [-]
A custom harness modeled after Pi, and we also evaluate solutions to one-shot coding problems
wslh 1 days ago [-]
[dead]
svara 2 days ago [-]
I haven't tried it for coding yet, but I ran a couple of my recent curiosity driven queries from Claude (mostly used Fable and Opus 5.5) through it.
This is a clear step change, I'm genuinely amazed.
Its answers are to-the-point, logically structured, clearly written. Like a really well made mini essay on your question.
It also really likes to build little simulations / models that you can tweak the parameters on. Amazing to explore how different assumptions affect economic scenarios, for example.
It's also great at picking (sourced) illustrations.
I'll be using this for everything that isn't coding from now on.
sebmellen 2 days ago [-]
It’s wild how good it is at explaining nuanced and complex health topics.
As an example, I recently had an OCT scan of my retina because I'm a “glaucoma suspect.” I'm a glaucoma suspect because I have an unusually high cup-to-disc ratio, which is a pattern typically seen in glaucomatous patients.
As part of this OCT scan, you get a report back on your RNFL, which is basically a report on the health and thickness of your optic nerve(s) and retina(s). (Dramatically oversimplifying, of course.)
My ophthalmologist struggled to explain my OCT scan results to me and instead sort of shrugged and said, "It's probably because you're a high myope, but we'll keep you coming in once a year.”
But my scans look horrifically bright red — meaning I'm in less than the first percentile of retinal thickness for my age. Obviously concerning enough to do my own research…
ChatGPT 6 Pro and Opus 5.5 have struggled to make any sense of the clinical implications of these scans. In contrast, Argon was the first model to:
1. explain how the OCT machine works
2. detail the physiological mechanism that causes the cupping and thinner RNFL layers in my scenario
3. sum it up into a clinical recommendation (similar to what my ophthalmologist recommended, with the addition of basic supplementation and lifestyle modification changes)
How do you know whether the explanation is correct or just sounds correct?
dt3ft 2 days ago [-]
You don’t. But its shiny and pretty.
KajMagnus 2 days ago [-]
Sometimes you can go to the websites of your country's health care system, look up what it said, read and compare.
api 1 days ago [-]
I'm an American and I might trust the chat bot more on some topics right now.
vanderZwan 2 days ago [-]
Would be nice if people just did that instead.
It's kind of infuriating to hear Dutch people cite these chatbots without bothering to check https://www.thuisarts.nl/.
clivedup 2 days ago [-]
Consider being curious why people turn to LLMs rather than just going to authoritative sources first.
Sometimes I don't know a topic well enough to even know the right terms, or sources. But an LLM can help me become familiar with relevant concepts, and then I can more easily find the authoritative sources.
Sometimes I don't know a topic well enough to know how far it is from my understanding. An LLM helps me quickly learn how closely I have the relevant foundational knowledge: if it's absolutely gobbledygook, then I realise I've probably bitten off more than I can chew; if I half sounds familiar, then I can more confidently begin independent exploring of authoritative sources.
vanderZwan 1 days ago [-]
Consider that I am not talking about people who use LLMs to learn about keywords to search for in other websites, but only regurgitate what the chat told them uncritically.
exographicskip 14 hours ago [-]
Well said! LLMs are excellent at translating my intent to what the domain's norms and definitions are.
The problem with google'ing 2 years ago was a combination of SEO hijacking and sheer ignorance on the user's end.
Critical thinking is a necessary ingredient regardless of the tools being used
Cthulhu_ 2 days ago [-]
Ironically I wouldn't be surprised if that site will become an LLM assisted chatbot soon enough. The Dutch health care system is under pressure and GPs (and their parent organizations, and the government) are reaching towards IT automation and self-help / pre-screening / etc to reduce the load. And granted, part of the load on GPs is by people who don't need to go there, who should just go to the pharmacy or stay home for a few days. But it also feels like seeking medical help is actively being discouraged, first by the "eigen risico" (copay), now by having to jump through hoops before being able to make an appointment.
vanderZwan 1 days ago [-]
> The Dutch health care system is under pressure and GPs (and their parent organizations, and the government) are reaching towards IT automation and self-help / pre-screening / etc to reduce the load
I'm aware, both of my parents are retired GPs.
Back when they retired, about a decade ago, they already had spent a decade complaining that the a constant increase in excessive administrative work got in the way of the actual work of being a doctor that helps their patients. So much that they had joined the strike (remember when that was a big deal because it happened for the first time in Dutch history?)
Anyway, the situation got more complex since then but a lot of pressure is also still administration. Yes, now we have assistants taking on more complex tasks leaving the "important" things to the real doctors, but I think that the need for that is also is in large part due to doctors not getting enough time to be doctors.
And I feel like automation will help fight back at that about as much as IBM's 1967 prediction came true that their office machines would fight back at the "paperwork explosion". Which is to say: Jevons paradox said no.
I mean, in this case it linked me to many sources, including the manual for the OCT machine used. One thing that even my ophthalmologist might not know, buried in the PDF manual, is that the sample size only includes 5 people under the age of 35 (skewing my "1st percentile" results more aggressively).
But you're right that you can't be 100% certain. Isn't that true for human-to-human explanations too?
Miraltar 1 days ago [-]
It's easier to sense how confident someone is in their knowledge
lanstin 1 days ago [-]
But the correlation between confidence and correctness, especially between different speakers, is famously missing.
vasco 2 days ago [-]
Many times you lack the context or words, but after it gave them to you, you can look up papers and figure out if it matches. Same as you do with human doctors to decide if you need a second opinion.
bonoboTP 1 days ago [-]
Yes. I guess most of the HN crowd can do this reliably, but most people cannot, and that's why doctors hate when patients google around and then lecture them based on a 5-minute WebMD skim.
SkyBelow 1 days ago [-]
If you have a solid self assessment of what you know, you can consider when information is supported by what you know. For the parts that aren't, you consider how critical they are, and identify the area that needs a deeper dive, where you either link it back to other areas you have confidence in your own knowledge or work to expand your own knowledge (using other resources). Then cross check knowledge. The cheap way you can do easily is to cross check between AIs/harnesses, because hallucinations happen randomly, so two different models in two different harnesses having the same hallucination is extremely rare. The more expensive option, which should be used when the information really matters, is building your own knowledge. Have the AI help guide you to resources, but not be the resource. While using non-AI resources, check you knowledge against them and the AI. Test yourself with classical resources where it really matters.
Part of it is using good judgment. Is the AI's claim reasonable if it came from another person? If some random person gave you a 2 page paper proving P=NP, would you even spend time to look at it, or would you simply toss it without caring to understand what error they made. Much like how scientists (and even the patent office) simply refuse to entertain any perpetual motion machines. Maybe if an expert in the field proposed it, and even that would depend upon how ("Hey, I found a source of free energy!" vs "Hey, I'm working through these equations and the energy term isn't balancing, can you help me find what energy source I'm missing?").
This does pose a problem. Those who are the best at understanding the limits of their knowledge and at teaching themselves need the AI's help the least but are in the position to be able to use AI with the best chance of not being misled. Those who lack these skills and would benefit the most from a personal educator, thus receving the biggest benefit in theory, are the ones most likely to fall victim to the errors made by the AI.
api 1 days ago [-]
How do you know that for a web site? You have to research and cross check, and talk to a real doctor.
BonerWiener 9 hours ago [-]
Easy. Just feed the results to a second frontier model to verify. /s
scotty79 2 days ago [-]
Ask a doctor?
clivedup 2 days ago [-]
+1. I've had successes from LLM-as-physio, but continuing to see the physio to make sure I'm not going mad.
"Just see a physio" misses that:
* I want the physio's advice during workouts
* I want the physio's advice during moments of pain
* I want the physio to explain things in more detail, but my 30-60 minute appt is not enough, and it's difficult/expensive to book them for longer.
convict 2 days ago [-]
Right, doctors love when patients bring their ChatGPT generated diagnoses for them to read over, it saves them so much work.
lp92 1 days ago [-]
He got a diagnosis from a real doctor. But like many specialized and highly educated people, they struggle to actually explain things clearly. Seems like Argon was able to explain it in a more understandable way.
scotty79 1 days ago [-]
It's not a diagnosis. It's a description of what a health problem means. If you go to the doctor empty handed and ask for explanation he'll love it even less than if you bring him nice visualization he can skim through and say, "yeah, that's about right".
Alex3917 2 days ago [-]
Yeah my personal experience has been that Gemini is much more useful for trying to understand complex health issues than OpenAI or Anthropic models. The standard of proof just seems to be much better calibrated, in terms of understanding that a potential treatment that is potentially dangerous probably needs more evidence backing it than another potential treatment without much downside risk.
sebmellen 2 days ago [-]
Absolutely. As one of my physician friends told me: “half the time, the best plan is, ‘We'll have to wait and see.’”
But, as a patient, it's hard to find comfort in that. I think that might be what drives the craziness in “non-traditional” approaches to treating disease. Could it be that people using fruititarian diets and essential oils for cancer are doing so because they don’t understand enough to bear a “wait and see” situation?
Argon explained the problem to me in such depth that I could be content with my basic treatment plan. ChatGPT went way off the deep end, suggesting things like high-dose niacinamide treatment (which is admittedly being evaluated in clinical trials, but comes with a host of other side effects).
binoct 2 days ago [-]
To be fair, try getting cancer and then consider how you feel about “wait and see” even when you understand enough to know it’s the advice with the best evidence.
sebmellen 1 days ago [-]
I know a number of physicians who have.
nl 2 days ago [-]
This is a great example of something that two years ago was unthinkable. I'm very impressed.
Thanks for sharing and I hope your eyes remain fine.
piyh 2 days ago [-]
I wonder how much that would cost to make in 2020 developer dollars
sebmellen 2 days ago [-]
I have no idea how I’d have even started on a project like this in 2020… certainly not by writing raw WebGL — ha!
disillusioned 2 days ago [-]
It's genuinely unthinkable... this sort of technology opens up a universe of bespoke micro-explainers and apps that meet you exactly where you are, for your exact, specific needs. I don't think we've ever seen a quantum leap in technological advancement that results in such a multiple orders of magnitude reduction in input effort related to volume and quality and type of output... ever. I mean, innovations like electric light, radio, television, the internet, and smart phones are huge transformative tools that similarly changed the fabric of what's possible. But this technology compresses work that would have taken pyramid-building effort and cost and condenses it down to... minutes of matrix multiplication for pennies worth of tokens. The mind boggles.
booty 1 days ago [-]
bespoke micro-explainers
Thank you for calling this out. There are certainly a lot of legitimate gripes with LLM generated code, but also a lot of never-before-practical solutions to manage code as well.
FWIW, I haven't seen the Gemini 4 explainers, but Codex and (particularly) Claude have been pretty decent at this for a while.
light_hue_1 2 days ago [-]
These are not explanations. They're visualizations. There's a huge gap between the two.
Explanations give you a mental model: when I move this parameter this the effect it has. This just shows you the obvious, make a thing bigger, it gets bigger on screen. That's not an explanation.
Models are quite horrible at explaining things. They produce local narrow views that don't take into account the mental model of the listener nor do they attempt to produce the kind of long-term knowledge gain that a teacher does. I taught for over a decade.
This is why models are extremely good for people who already know what they're doing. That narrow why is all you need, like, tell me the 3 sentences that describe the mechanism behind this paper, and I'll get the rest. But building the scaffolding onto which to hang a lot of knowledge?
Give it a try. Try to have a model teach you something new. Not like "how do I tie a knot" but like a new area of math where you're going to dedicate 3 months of your free time to learning it. The results are pretty horrible.
avilay 2 days ago [-]
+100 I tried to get Claude and Gemini to build me a course on probabilistic ML from scratch. They both failed. This was a few months ago. The best they could do was design a course curriculum. My learning since then has been through asking the model to explain a very narrow topic from the curriculum. And then go through the agonizing back-and-forth questioning and answering to actually build an understanding. No doubt the models are great at this and my learning has never been so through with such solid foundations. But zero-shot it was not!
arational 2 days ago [-]
Isn't prompting a full course in one shot similar to prompting a full fledged app? Aren't we supposed to break it down into plan mode vs execute mode?
apsurd 1 days ago [-]
But that fits with the point that you’d have to already know the domain to be able to break it down into discrete units for steering the LLM.
imnotr0b0t 2 days ago [-]
I fully agree. I tried using several language models (LLMs) to learn Chinese. It turned out to be a so-so idea.
terribleperson 6 hours ago [-]
Did you throw it into ChatGPT 5.6 pro? Some people claim 5.6 is smarter than 6.
Melatonic 2 days ago [-]
Guessing they've been going all in on the research angle for health stuff and might have an actual product for medical professionals soon
A_D_E_P_T 2 days ago [-]
Totally agree. This is my new favorite general-purpose/non-coding model. It's surprising how good this thing is... And fast, too.
informal007 2 days ago [-]
It’s pretty slow for me. I’m not sure if it’s due to I trigger an agentic task by Gemini website entrance.
Google have strict limits on new model this time and are pretty conservative. As we know they spent lots of time on internal testing since release.
All the signs mean Gemini 4 argon is a pretty large model than previous.
bitexploder 2 days ago [-]
It is a very efficient model, which I really appreciate. It can definitely feel a little slow. I tend to not hang around for it to do its thing, but for orchestrating and taking somewhat vaguely defined coding goals it is pretty amazing.
where are you guys playing with it - i logged into ai studio and it's a mess. cant event figure out what model im using
A_D_E_P_T 2 days ago [-]
gemini.google.com with a $20/month Pro membership.
NortySpock 2 days ago [-]
It's available via OpenRouter
holaysuns 2 days ago [-]
Google's models have usually been pretty good at general query stuff, they fall flat when doing harder tasks.
HSO 2 days ago [-]
> I'll be using this for everything that isn't coding from now on.
why? is it not so good on that? or is there something better performance, or better price for same performance?
A_D_E_P_T 2 days ago [-]
I'm not the poster you responded to, but I can tell you that historically the Gemini models have lagged behind Anthropic (the clear leader) and OpenAI's best models where coding in particular is concerned. Opus 5.5 is, right now, the consensus best coding model.
That said, I'm extremely impressed with Argon thus far. I take back everything bad I've ever said about their AI strategy. Argon is very good and very fast. It might be competitive at coding, but it's too early to say.
chrisweekly 2 days ago [-]
Opus 5.5 medium is a real sweet spot for many many things. But. I got an unexpected refusal (for help creating a browser console script to convert a web-based streaming video to an offline-ready .mp4), which Opus 5.5 low happily provided.
complex_fir_rea 2 days ago [-]
Since Google owns YouTube, this refusal by Gemini seems suspiciously too aligned with their interests. Just a conspiracy theory, though :)
wnm 2 days ago [-]
it does, but the refusal in OPs story was by Opus 5.5 medium, I think :)
chrisweekly 1 days ago [-]
Correct. Also, it wasn't a youtube video.
sharken 1 days ago [-]
Actually the opposite, Opus 5.5 medium did the job.
Anecdotally, I have had Opus 5.5 decline a reverse engineering effort into a Zigbee gateway that I own.
DeepSeek did not have any issues with this using Claude Code as harness.
chrisweekly 1 days ago [-]
My comment was about an Opus 5.5 medium refusal, for a request which was then satisfied by Opus 5.5 low.
sharken 11 hours ago [-]
Interesting, might want to try a different model the next time.
fonrex 2 days ago [-]
I completely agree;
I was really surprised to find that Claude Code Opus 5.5 is significantly better. I had spent quite a bit of time using Gemini Pro for coding tasks, but when I revisited the code analysis using Opus 5.5, it revealed quite a few shortcomings in Gemini's understanding...
I code using Python, which is considered the best language for writing code via LLMs (I am referring to linguistic entropy).
senderista 2 days ago [-]
Opus 5.5 routinely finds tons of bugs in code that Opus 5.5 wrote, so this may not reflect a difference in model capability.
dieortin 2 days ago [-]
Sounds like you were using the older Gemini 3.1 Pro from February, and not the new release this post is about
charm137 2 days ago [-]
I am using it via OpenRouter for going through some Vulkan code. It's a beast - it has no hesitation putting in instrumentation within the code (having a number of render passes) to dump the framebuffer to PNG files and doing pixel-by-pixel comparisons, putting in false colors in shaders to identify bugs etc. Very impressive.
kccqzy 2 days ago [-]
That’s something Opus can also do. Perhaps by coincidence I also recently debugged shader bugs where Opus dumped the resulting render to PNGs and then using Python PIL to do comparisons. It purposefully added false colors too.
Topfi 2 days ago [-]
Suspect there is a training set going around focused on comparing and extracting data in distinct steps from image input. Saw it first on Opus 5 on a backpack construction task to extract and compensate for a reference tape being slightly offset in a LIDAR scan, followed by heat maps for per pixel comparison between the reference and blender render. Opus 5.5 did go a step further in the same, extrapolating offsets to compensate for capture inconsistencies. Some may not even notice this happening as Claude Code steers the model to use the scratchpad hiding much of this work.
Found this especially impressive because backpack material related research has been something I know models can still get caught up on and the models being decent at web research in coding nowadays does absolutely not apply to these task equally Recommending nylon canvas shell suppliers or waxed cotton when PVC coated cotton is explicitly request, that sort of thing. While not perfect in reading coding docs, models are far better there.
Like with front end and code “parsability”, feel like you can very clearly point at which labs use what training set, with many often sharing the same suppliers. Once I noticed that, I became very bullish on specific training sets over emergence as the source for current gains.
guyomes 2 days ago [-]
Out of curiosity, which harness do you use for this kind of task, and how do you sandbox it? I used Aider before, and consider switching to a more recent tool.
charm137 2 days ago [-]
I started out with agy CLI but it's horrible because of its very indeterminate context compaction. But I live with it because it's the only way to use Google's subscription. I was and still use Opencode in parallel (accessing Openrouter ZDR endpoints) when I run out of rate limits / weekly limits.
Over the past two weeks, I have started using a locally modified version of deepseek-harness (modifications include disabling telemetry and querying a provider's model list dynamically through Openrouter, instead of relying on the pre-packaged pi-ai manifest that comes with the base code). So far, that has been the best experience - I really like the caching, tokens/sec stats that it provides. (EDIT: I don't know a lick of TypeScript but AI-driven modification of the harness is good enough for me to make these tweaks).
Sandboxing is a definite issue for me. As you perhaps know already, headless operation is often not an option in graphics/game programming which requires actual render-surface instantiations from windowing systems and then input-context handling (keyboard/mouse/gamepad) as well. So I run most development on a VM. But part of my test-suite and smoke-tests rely on runninng it directly on the host OS (which is not ideal). I'd love to hear suggestions to avoid this.
gpderetta 2 days ago [-]
> So I run most development on a VM. But part of my test-suite and smoke-tests rely on runninng it directly on the host OS (which is not ideal). I'd love to hear suggestions to avoid this.
I'd recommend asking the harness to modify itself though (or a cloned repo in another directory) rather than applying the above.
2 days ago [-]
julianz 2 days ago [-]
Inexplicably, despite the fact that I pay Google actual money for an AI plan, they don't want me to use their latest model. Instead, I'm allowed to use the competition (Opus and Sonnet 5.5). I don't think it makes sense but what do I know?
heyjamesknight 1 days ago [-]
I think the rollout was phased. I couldn't see Argon in Antigravity on my personal AI Pro account until late last night.
etoxin 2 days ago [-]
I'm using Gemini Argon 4 now with antigravity cli and it can absolutely code.
tobias2014 2 days ago [-]
When does it auto compact? Is it at 250k tokens as for gemini-3.8-flash? If it's more generous I might consider upgrading to the ultra plan.
etoxin 2 days ago [-]
it has a context of 1 million tokens. Antigravity classifies the context which is helpful.
vardalab 2 days ago [-]
do you have ultra plan? it says regular sub doesn't have it.
etoxin 2 days ago [-]
i'm using our companies enterprise account.
akoboldfrying 2 days ago [-]
> I haven't tried it for coding yet
edstarch 2 days ago [-]
Big if true. Opus 5.5 is a lot better than 5.0 but I find it maddening thay no single model released in the past ~year can write prose with a clear and coherent voice. I blame RL on programming tasks.
AspireOne 1 days ago [-]
Yes, that's the reason. Anthropic said as much in their statement (they said that there's a tension between writing human-understandable prose and prose optimal for other agents to read).
As a side note, I am kind of sad that basically all models nowadays are RLd to hell and back specifically for coding and deep reasoning, and as a result, they lack in creative writing, human-readable prose, and even general knowledge and abilities (for example, qwen 3.8 27b is a beast for coding, but reportedly falls off the cliff in anything else). Gemini 4 Argon seems to be an example of what you get when you DON'T do that - Google seems to be one of the only labs optimizing for general performance.
I also fear that models cannot be excellent in everything at the same time. I fear that there will always be a heavy tradeoff between what specific field the labs optimize for VS everything else. I don't know.
heyjamesknight 1 days ago [-]
In my personal testing, Argon is exceptional at writing tasks. After a few simple eval loops of you correcting things into your own voice you can have it sound almost exactly like you.
drob518 2 days ago [-]
My first reaction after reading the announcement post is that the coding scores look like a weak point.
da_chicken 2 days ago [-]
Google is clearly not targeting coding. They don't have to. They're Google.
scarmig 2 days ago [-]
Coding is probably one of the worst value propositions, in terms of consuming more tokens per subscription dollar. Doing well in coding might even hurt Google's revenues: it would cannibalize some proportion of Anthropic's user base, which Anthropic serves by... renting Google's infra.
bitexploder 2 days ago [-]
Gemini Flash 3.8 was heavily optimized for agentic coding. It is not an ignored use case. Flash 3.8 benches extremely well on agentic coding. It was very intentionally made good at coding. So was Argon. It is doing a lot of mission critical porting work for things like Rust. Does Google also build models for things other than coding? Yes. It has a lot of enterprise customers that need good general models too. IDK, I haven't seen anything convincing in Flash 3.8 or Argon that say "Google is ignoring coding" it just took a while to get there since Pro 3.1.
(Disclaimer: I work at teh Goog and spend... a lot of time with these models :)
scarmig 2 days ago [-]
Quota allocations are a massive decision that go all the way up to Sundar: what GDM organizationally wants (training) vs what GCP wants (Anthropic to throw it money hand over fist) are not aligned and a balance has to be made. For various reasons, GCP carved out a significant share; an eleven figure check over a quarter or two carries a real weight. This doesn't mean that GDM was starved for compute or that it wasn't making significant progress, but it absolutely could have had a more impressive H1 if given more resources.
drob518 1 days ago [-]
Am I misreading the Argon numbers then? It leads on DeepSWE and Vibe Code Bench but lags on FrontierSWE and Terminal Bench. Should we be viewing Argon as a frontier coding model?
4dd3aa 2 days ago [-]
Well done - youre the first poster Ive seen on here who delivered a short-sharp-nuanced analysis on this topic! Well done! (not sarcastic btw)
Wolfenstein98k 2 days ago [-]
Wouldn't it be more profitable to have the user pay you for your model, running in your servers, than for a while separate company to try and make a profit out of being the middle-man?
I mean you'd have to crunch the numbers, but Anthropic would need one helluva advantage in efficiency before that would be cheaper to google than the full stack in house
pingou 2 days ago [-]
There were rumors that the reason Gemini was delayed was because it was too bad at coding.
VirusNewbie 2 days ago [-]
I've found it on par with Opus 5.5, what don't you like about it for coding?
svara 2 days ago [-]
I just haven't tried it yet. Also I'm used to Claude Code now so it'd need to be a lot better for me to switch to a different tool.
transitivebs 1 days ago [-]
"curiosity driven queries" – love this phrase, i'm def going to start using it :)
jwpapi 2 days ago [-]
I’m curious about your curiosity driven queries. Mind to share?
dgacmu 2 days ago [-]
I'm not the person to whom you're replying, but I just tried one of mine:
"I'm interested in understanding the rate of VOC dissipation from fresh oil paint on a surface versus the ventilation rate in the room in ACH. I would prefer solid academic studies of this, followed by analytical evaluation, followed by references to non-quantitative sources. Tell me what's known about this."
that I had run in ChatGPT earlier this week. (Yes, we just had our bedroom repainted, and it's TERRIBLE and I hate it, why do you ask? :-).
I didn't try it first in Gemini Flash, but I did try it in whatever model is used for Gemini Live and the answers were terrible. The ChatGPT answer (I think on medium) was pretty good but it threw in extraneous formulae that weren't actually germane to the analysis. The Argon 4 query I just ran was honestly the best of the answers I've gotten -- it was clear and really well-structured from the perspective of "here are the three big phases that researchers break down paint drying into and what the differing effects of ACH are on those phases".
I'm totally going to give this one a try, after basically giving up on 3.8 flash for most things despite having a free $20 pro subscription (from buying a phone).
Lately, I've been having a lot of Socratic conversations with agents to understand these kind of topics, this helps me a ton, thank you so much!
Melatonic 2 days ago [-]
Kinda odd there's no mention of room humidity.....
bjoli 1 days ago [-]
I am a professional bassoon player. I asked about multiphonic fingerings. It could actually suggest corrections for the fingerings that did not work.
adventured 2 days ago [-]
With 3.8 Flash on High I noticed a huge upgrade in intricate discussions related to physics, material properties, real-world application, and so on. Things that I've had a hard time getting LLMs to be able to discuss accurately. They often discuss textbook knowledge fine but can't apply it properly or can't think outside the box at all. 3.8 High understands applied material sciences in a way that GPT Astra flunks at often. I asked Gemini to explain that result, and it said basically that GPT was merely book smart in a classical sense and that it was likely undertrained on more obscure material.
lp92 1 days ago [-]
I've noticed the same thing with 3.8 Flash vs Opus for aero model generation.
Melatonic 2 days ago [-]
Any examples ?
walrus01 1 days ago [-]
I have been rather impressed with the depth and breadth of knowledge in a Q8 of 3.8-flash-next which fits in under 200GB RAM, running, with a 16384MB extra cache size.
hardbass 2 days ago [-]
I have had the google search results ai also do these simulations and code generations unprompted, I thought that feature already existed.
pjm331 2 days ago [-]
I have a project I am working on where I wanted to have nice docs, and so I have some slop docs sitting there as placeholders until I get around to writing them, awful claudish stuff
I've actually spent some effort trying to make them halfway decent placeholders by putting together a skill based on diataxis framework (https://diataxis.fr/) that at least got everything into the right location
just took a pass at rewriting them w/ argon and am very pleased, big improvement
gigatexal 2 days ago [-]
Just praying they don’t nerf it
mwigdahl 2 days ago [-]
Agreed, its communication abilities are very good. I'm a big fan of Opus 5.5 but Argon just flat out writes better.
halJordan 2 days ago [-]
[flagged]
noumenon1111 2 days ago [-]
Which reminds me, where is @simonw ? lol
Jimmc414 2 days ago [-]
Generating pelicans on bicycles most likely
glouwbug 2 days ago [-]
We have this same point made every release. I wonder if models actually are getting better or if these companies are just secretly hamstringing the previous models before releasing a similarly performing model
taneq 2 days ago [-]
I think it takes a while to start finding the weak points in a model. It seems fantastic until you use it enough to find out that it hallucinates APIs, or can’t count letters, or uses too many em-dashes, etc. you don’t really know someone (or something) until you’ve lived with them (or it).
gazebo2 2 days ago [-]
I think there's definitely some quantization or nerfing happening post-release -- I definitely noticed that with Gemini Flash 3.7, it was super fast and accurate at the start and a few weeks later seemed to run in circles and overthink even a "what are you doing" message, all while doing almost identical work. I find it weird that we're so far into the LLM era and the post-launch degradation is still something of a conspiracy theory.
WillPostForFood 2 days ago [-]
Models are actually getting better.
KerrAvon 2 days ago [-]
While IME 4.7 and 4.8 were not appreciably better than 4.6, Opus 5 was noticeably better, and Opus 5.5 is significantly better than Opus 5.
becquerel 2 days ago [-]
Try using a model from a year ago.
selcuka 2 days ago [-]
What would that prove if we are testing whether they "are just secretly hamstringing the previous models" as GP claimed?
staticman2 2 days ago [-]
This would require a conspiracy since Claude models are hosted by Google, Anthropic and Bedrock. All these companies would have to be working together to secretly hamstring the previous models.
selcuka 2 days ago [-]
I'm not claiming that they do, but just to play the Devil's advocate:
Anthropic controls the full serving stack, so it can change precision, hardware, kernels and settings whenever it likes. Reducing the KV cache precision, for example, saves a lot of memory (and money). It may even be possible to quantise the model on-the-fly when loading it into memory. They don't necessarily have to upload a whole new model.
Even if they can't do the same on all platforms, we can't deterministically compare them anyway.
staticman2 2 days ago [-]
For the Bedrock and Vertex versions the customer is doing business with Google and Amazon directly. I doubt Anthropic's contract allows them to unilaterally "change precision, hardware, kernels and settings whenever it likes."
At any rate my point remains that this would require a conspiracy between 3 companies.
sdenton4 2 days ago [-]
Welcome to the hedonic treadmill! Just keep walking!
postalcoder 2 days ago [-]
I'm using this model via API. I am most excited about its ability to write the english language (or any language). My least favorite trend on twitter has been people posting 2D illustrations/3D renders as a proxy for model capability. Well-written prose is far less likely to go viral but far outstrips the importance rendering a fantasy world in one shot.
I've been trying it out via the API and the model has been a delight so far. Clear in the head, not RL-fried, distinct writing style, and big model smell. That said, the em-dash is back w/ this model.
That said, I would be so frustrated with Sundar Pichai if I was on the Google team. The botched rollout of this model seems to be entirely a matter of capacity, and Pichai has been more than happy to give it to anybody but his own team.
asdfman123 2 days ago [-]
I've been using it for a while internally at Google. I prefer Argon over Opus 5.5 because of the clear thinking and directness. It phrases things in an organized, straightforward way.
andai 2 days ago [-]
Interesting. My favorite LLM output ever was the reasoning traces in gemini 2.5 pro. I found them much easier to understand than the "final output", because they were very structured and concise. Of course, they ended up hiding the reasoning, not long after that, like everyone else... sad!
onevsall 2 days ago [-]
Same here, especially the march edition. It was incredible. I thought Google will run away with it.
I haven't tried Argon, but for writing, I actually prefer 3.8 Flash over any GPT and Opus.
petesergeant 2 days ago [-]
If true, this will be a gigantic departure from any model Google have released to date. I’m hopeful but also pretty skeptical.
asdfman123 2 days ago [-]
I feel Argon is about as good as Opus 5.5 but I find Opus annoyingly sarcastic and incomprehensible.
I know it's a matter of personal taste but it reminds me of a grad student that tries to sound smarter than he is by being dismissive and opaque.
flockonus 2 days ago [-]
Opus 5.5 has the personality of a proud righteous nerd, definitely incomprehensible at times, unless you explicitly tells it to slow it down, it crams information far too dense.
Great model to architect and resolve eng. problems in a harness loop tho. Poor handoff.
FranzFerdiNaN 1 days ago [-]
So its clearly trained on Stackoverflow.
rad-b 2 days ago [-]
Yes, exactly! Maybe not sarcastic but definitely using jargon and being opaque like we all know it, it’s just hinting. Feels like talking to someone who’s constantly trying to prove to me they are smart(er).
chpatrick 2 days ago [-]
The thing is that if you do know what it's talking about then it gets you the information the quickest that way. If you're not an expert you can always ask it to explain. I don't want it to ELI5 by default.
elcomet 2 days ago [-]
That's not my experience, the model is adding tons of unnecessary details, that make it much harder to find the important information for an expert that already knows all those details.
I don't want ELI5, I just want the model to go straight to the point, like a colleague human expert would do.
chpatrick 1 days ago [-]
I don't know, that's exactly what Opus 5.5 feels like to me.
rixed 1 days ago [-]
Not my experience either. I find it both verbose and obscure, in its comments or explanations. It also oftentimes assumes that I know what I don't and that I ignore what I actually know.
mattlondon 2 days ago [-]
Agree I do not like the way Opus 5.5 communicates with me. It feels like it is trying to act superior in a condescending way. DO NOT LIKE. I have not used Argon yet but Gemini 3.8 Flash has been a joy to use in comparison (and so fast!) as a daily driver.
IndySun 2 days ago [-]
>...trying to act superior in a condescending way...
Strange. I wonder why? Have you adjusted 'roles', added 'ask clarifying questions'? This is most peculiar.
geertj 2 days ago [-]
Yes! Literal copy paste from my session with Opus 5.5 this morning:
> You gave me a long answer but it didn’t answer my question: can we safely remove $X
glhaynes 2 days ago [-]
Sarcastic? wdym?
mkoryak 2 days ago [-]
Opus has no sense oh humor at all. I wish mine was sarcastic.
bitexploder 2 days ago [-]
I have used it as well. It is very dense, but it cites with evidence and has a direct and straightforward approach and manner. It doesn't really have a "tone" or mannerisms the way Opus 5 did. It follows instructions and sticks with pretty clean language by default. Argon is solid and handles long horizon well.
cluckindan 2 days ago [-]
Em dashes are underappreciated — they are visually pleasing, not cluttery like semicolons or definitive like colons.
bitshiftfaced 2 days ago [-]
I have a pet theory that there's some overlap of writing patterns that A) indicate quality, clear communication and B) those that the LLMs have been converging on. As a result, we're ruining the reputation of some great writing patterns because we've associated it with AI slop.
TeMPOraL 2 days ago [-]
My pet theory is that this was happening, because labs naturally worked to source highest quality data sources they could get. Even if they initially slurped everything indiscriminately, the obvious next step is to sort them by "quality", and bias towards the high end while trimming from low end.
So obviously, government documents, top publications from liberal arts scholars and STEM > Wikipedia > Reddit (and within, r/AskHistorians > r/all, etc.). The first group is the one most likely to (over)use correct / fancy typography.
I have a related pet theory: the telltale sign of AI writing is being too good for the context it's found in, in terms of vocabulary and style as well as structure.
What is jarring about this is precisely the quality - we associate certain styles with important (or self-important) venues, so it's uncanny to suddenly see random shitposts, copywriting spam, or regular chat replies look like work of NYT bestselling authors or legal scholars - it feels wrong in places we expect basic language, grug-brain reasoning, and lots of typos.
Also, what makes the above much more noticeable, is that AI systems don't have cross-session stylistic memory. From the POV of an LLM, that perfectly fitting, masterfully crafted starter is the first and only time it used it. From the user's POV, it's the tenth time they're seeing it in the span of an hour.
solyin 2 days ago [-]
There's also no voice. If I get a text from a friend, I can hear them. Their tones, their mannerisms. I can't hear Claude because it's not a person that I've met and interacted with.
InvisibleUp 2 days ago [-]
Would that not also apply to most books, newspapers, and internet comments?
Retric 2 days ago [-]
People use language in their own distinct fashion. To the point where you can often figure out the writer from a long enough sample. Newspapers and internet comments end up a blend of different voices but in visually distinct boxes.
LLM intentionally randomize output to cover up all the plagiarism, but this largely destroys the consistency that cause writers to feel unique. Larger patterns still emerge but it just feels stale.
Christine725 1 days ago [-]
Models are of course different from humans, because the process by which they were created is different. But what’s undeniable is that humans also love to quote or repeat what others have said — we learn from our own kind, that’s how we gain experience. Yet rarely does anyone hold a person to a word-for-word standard over the course of their entire life, checking how many of their sentences were picked up from other people’s conversations — and then call that infringement. Just as no one right now would say I’m plagiarizing a model’s speaking style because I use em dashes, and no one would consider it misconduct when I reference a famous person’s ideas in everyday conversation. I don’t think models intend to infringe; it’s simply how they learned language. But perhaps, just like humans, they need to learn to provide more citations for what they reference.
Retric 23 hours ago [-]
Even individual words can be an issue. Humans understand when you want a new story about a magic school there’s issues using Hogwarts as the name.
LLM’s will happily regurgitate multiple paragraphs practically verbatim if you don’t specifically design them not to behave in this way.
Because ultimately when you design a system to best predict the next output based on a given input perfect replication of prior training is the correct response mathematically.
TeMPOraL 1 days ago [-]
That's a misconception. LLMs are trained to suppress regurgitation, not because of plagiarism concerns, but because quoting training data verbatim means the model didn't learn from it, just memorized it.
Models absolutely do have voices. Spend some time using them, and you'll see. I can easily tell Claude-written prose apart - a consequence of using it daily.
Retric 1 days ago [-]
You misunderstood, “Larger patterns still emerge” means yes you notice LLM’s but it’s for different kinds of patterns than you get from people.
> not because of plagiarism concerns
I’ve second hand accounts of meetings focused on exactly the problem of how much verbatim text was being spat out and what to do about it. Some versions of specific models are far worse at this but it’s a huge legal liability not to address it.
freecodeio 2 days ago [-]
can't say the same for newspapers or internet comments, but when reading books there's usually a voice or a "vibe", and if there isn't one then it's just an unreadable book
astrange 2 days ago [-]
AI writing style comes from post-training, which is entirely under the control of the labs. It's not copying it from any public sources.
TeMPOraL 2 days ago [-]
I didn't say it's copying anything. It's learning styles - all of them - in pre-training. Post-training is just about giving it the right defaults.
Doesn't change my point anyway - even if post-training, labs naturally want the LLM to come off as smart (well, except for Grok). Post-training is where they "untrained" em-dashes and such, due to complaints of users.
xienze 2 days ago [-]
I'd heard an interesting theory that in training on customer chats there's a lot of interactions like this:
Agent: _wall of text_
User: yeah sounds great, do it
Which has the predictable outcome of biasing towards wall of text responses.
xd1936 2 days ago [-]
Continue to write with em dashes. Be the change you wish to see in the world.
lackoftactics 2 days ago [-]
I did at the beginning, but there is only so much you can take, and no one cares if you have read books and seen it before. There were a bunch of programmers whom I thought would understand em dashes, but they did not and said no one uses such dashes only normal hyphens.
Em dashes ended up as one of the first victims of llm age
TeMPOraL 2 days ago [-]
Em dashes were already "doomed" at least half a century ago, because they didn't make it into ASCII character set - which arguably is just because they were seen as redundant on ancient typewriters; that's also why they aren't there on normal keyboards, which was a double whammy: inconvenient for programmers to support, inconvenient for users to type.
In other words, since the very start of computer era, the use of em dashes was always a signal.
jamie_ca 2 days ago [-]
At least with a typewriter, if - was sized right you could -- for an emdash.
dagss 2 days ago [-]
I have used -- for decades.
onevsall 2 days ago [-]
It is not because those patterns have no use. It is because LLMs overdo it.
Every point is delivered with grandeur and rhetorical flourish. It is hard to reward 'negative space', a simple writing, A to B, vanilla pattern.
Melatonic 2 days ago [-]
I think it's because it starts writing the final output and doesn't have the ability to go back and correct. It's like a stream of consciousness bandaid
epihelix 2 days ago [-]
They were in models in the first place because they are used ubiquitously published formal writing. As, of course, are semicolons. There is nothing wrong with using punctuation for nuance and structure.
knollimar 2 days ago [-]
I feel like the LLMs used them "wrong". They use them for hard breaks for disconnected thoughts rather than one thought that disconnects.
epihelix 2 days ago [-]
I think they just used them too much. I suspect that as they naturally form a thematic break point in writing, their probability was unnaturally high at the end of phrases and this led to overuse.
I don't ever recall seeing an LLM using them incorrectly, as opposed to simply at high frequency but correctly (although I'm sure some low param or early models must have done so at some point).
dalemhurley 2 days ago [-]
Em dashes are commonly used in most countries that speak English. I get the feeling that people from the USA don’t use them often and now have tarnished the use of them.
mikestorrent 2 days ago [-]
Do you remember when the AIs were all writing with bullet points all the time? I miss that! I'd prefer that to the current more-information-density-than-any-human-would-ever-muster approach where sentences are packed so deep you can't get any meaning by skimming.
nostromo 2 days ago [-]
The three great epochs of AI:
Way too many bullets -> way too many emojis -> way too many em dashes
Melatonic 2 days ago [-]
Hopefully they stay away from my damn tilda !
qlte 2 days ago [-]
Like many other LLM tropes they were heavily overused which attracted attention. Normal English writing doesn't average 1 em dash per 2-3 sentences like I would often see during Peak Em Dash.
They aren't a complete rarity by any means but still relatively uncommon compared to the typical punctuation one sees in routine usage.
I loved em dashes back in high school and my English teacher would have definitely returned an assignment with a lot of red marks if I used em dashes as frequently as an LLM (speaking from experience).
pohl 1 days ago [-]
Most complaints about AI writing are people telling on themselves for not reading broadly.
holoduke 2 days ago [-]
The dashes reveal AI generated text. For me a trigger to further ignore the text.
xandrius 2 days ago [-]
Because you ignored them before someone told you about them.
Yizahi 2 days ago [-]
See, the problem is when em-dashes are used like you just did, they are fine and even expected. But LLMs use them differently. Here is a fresh example, from today:
"Binds the downstream RF port template—which we configured earlier—to this RPD profile, applying all global DS settings and channel structures to the node's downstream hardware."
Do you see the problem in comparison? The LLM text is assuredly not how humans write. Well, maybe you would find similar examples in the classic literature, but definitely not in a common internet texting or simple articles and posts.
michaelbrave 2 days ago [-]
in all of my writing I use parenthesis in basically the same way (you know to explain the addendum of information).
jurnalanas 2 days ago [-]
can confirm this. I didn't have high expectations for AI writing good long-form content in Indonesian. However, I recently tested it on some of my Indonesian writings and asked Argo to rewrite them while maintaining the original tone—and it absolutely nailed it.
algoth1 2 days ago [-]
I missed the em dash — ngl
leoooodias 2 days ago [-]
As a longtime user of em dashes, I refuse to change my style because bots happen to agree with my choice of punctuation.
johnisgood 2 days ago [-]
Well, I have to change the way I write at times because my comment gets automatically flagged. :/ Not fun.
gfody 2 days ago [-]
this happens to me as well, meanwhile Pangram claims their false positive rate is 1 in 24000 as if that should be practically negligeable. I bet it's closer to 1 in 10 for gen-x'ers
braj0 1 days ago [-]
Isn't that, potentially, a form of discrimination?
Quarrel 2 days ago [-]
It cops hate because people try to pass off LLM written text as their own, then get pissed when called out on it.
If you just want good llm text, it’s great.
thmorriss 2 days ago [-]
me too. it's like a fingerprint of ai. I think ai should be proud to be itself and be ai.
kypro 2 days ago [-]
Yeah, I never got the hate. If it reads well who cares if it's not something a human does much?
I think there was a lot of misplaced hate of the em-dash due to all the low-qualify AI slop people were trying to pass off as their own for a while.
algoth1 2 days ago [-]
The worst part is that it seems they just regex it with the semicolon, making the text harder on the eyes
andybak 2 days ago [-]
> Well-written prose is far less likely to go viral but far outstrips the importance rendering a fantasy world in one shot.
I'm not entirely sure you're correct. Your claim boils down to "Writing natural human prose" vs "Broad use of complex software and spatial awareness". My gut feeling is that there's more untapped and fruitful applications in the second group than the first - at least in the short term.
2 days ago [-]
pkulak 2 days ago [-]
I was kinda hoping I'd get to the end and it would say, "In fact, this entire post was drafted by Argon!"
polynomial 2 days ago [-]
> far outstrips the importance rendering a fantasy world in one shot.
Given FelonyBench, do we also have a FantasyBench?
postexitus 2 days ago [-]
What was botched about it?
postalcoder 2 days ago [-]
Announced the model two weeks ago to a ton of hype, model was unavailable for anyone to try. On twitter, Google folks went on a great run taking shots at Muse ("Muse who?"). The hype died down bc nobody could touch the models and meta/muse came out with a similarly capable model in the meanwhile.
At the same time, OpenAI had a disastrous dev day that led a lot of users to jump ship to Claude.
Couple weeks later, Gemini-4-argon is still unavailable on anything less than the ultra tier, which means there's no natural funnel to get ppl to pay for the more expensive ultra plan other than "trust us the model is good".
If Google had argon broadly available earlier this month they would have picked up a ton of new users.
asdfman123 2 days ago [-]
As someone internal at Google I don't understand why there's so much criticism of Google models... but right now I'm using the next model beyond Argon, and Flash 3.8. In my personal Gemini app I only have access to 3.5.
Models move so fast now and Google is very cautious about releasing them.
vessenes 2 days ago [-]
My complaint, and the complaint of many is not the models - they can be good to very good depending on use case.
What sucks, hard, is googles ability to deploy direct to devs and customers at scale when the deployment is not a longstanding Google project.
Case in point: I’m an ultra subscriber and see in my app right now an ad to try Gemini 4; meanwhile the only models I have available are 3.1 pro,3.6 flash, and 3.8 thinking.
Clicking the try gemini 4 link takes me to a demo prompt that loads 3.6 flash. Srsly
I won’t even start telling you what getting api key based access together was like a year ago.
avinash147 2 days ago [-]
[flagged]
michaelbrave 2 days ago [-]
I think google makes great models, my favorite models have been google models. My critisisms are with the infrastructure around the models (or lack of them),things like billing, access, changes to terms and limits, pricing etc. All of these have shown either inconsistency, lack of care, sometimes confusing offerings, and things that felt like rug pulls.
Please google, just make it easy for me to give you money and get what I paid for.
justinhj 2 days ago [-]
People have very short attention spans and cheerlead whoever is winning at any particular point in time.
meowface 2 days ago [-]
For agentic coding, Claude Code with the latest Fable or Opus is still just better. That's what people care most about.
Unless Gemini CLI greatly improves and Argon works well in it, this probably won't make a dent in the coding market.
BoorishBears 2 days ago [-]
"I don't understand why people don't like our models but we can't get our act together on the products they use those models in"
asdfman123 2 days ago [-]
Uncharitably misreading things on the internet and getting upset about them is not the best hobby
user_7832 2 days ago [-]
I agree that reply was unnecessarily rude, but... idk man. I'm a huge cheerleader for Google [1], use 3.1 Pro via AI studio, I'm on a Pro plan... and I can't access this model? Not even a "4 chats per hour" type thing?? I mean, I have video generation access for some reason (no good for me), but Google's policies are baffling to say the least.
Ofc I know this isn't your fault in person, but I really wish Google would get its shit together.
...As a long time tech enthusiast I should probably stop being so delusional. Heh (cries internally)
1: lol I know who cheers for a trillion dollar company but I'd rather not cheer for OpenAI
asdfman123 2 days ago [-]
Yeah, what I was trying to say is "the criticism doesn't make sense to me because I forget because I have access to the good stuff."
As a Google employee my defense is it's hard for a company this big to move fast, but I can understand how Google customers would be annoyed.
BoorishBears 2 days ago [-]
There's nothing unnecessarily rude about my comment, they are literally unable to get their act together on product.
The top level comment blamed cheerleading but let's apply critical thinking for a second as product people and look at the funnel:
1. Potential users sees new release
2. Potential user takes time to install and set up Google harness (you already lost 90% of people here, the funnel better close quick!)
3. Potential user cannot use new model.
Then what? You hope they developed good will towards your model through this experience?
You stuck a two week wait in your funnel? Is that the definition of not having your act together or what?
We're on HN, how many of us have heard from Google staff this kind of stuff is a result of them growing so big that AI Cloud has to compete with Vertex Cloud, and AI cloud exists because Vertex Cloud needed 2 years to get OAI/Ant style API keys working and were losing dev mindshare in the meantime?
vardump 2 days ago [-]
Didn’t check agy or AI studio, but at least Argon is available in Gemini page/app.
BoorishBears 2 days ago [-]
Why are you doing that then?
2 days ago [-]
varispeed 2 days ago [-]
Because Google models are crap. They are on par with local shit models. Last time I tried they were unable to even hold a longer conversation, because they would enter loop and reiterate last answer regardless of the prompt etc.
Then basic mistakes (inferring wrong information from the given text, hallucinations...). Gemini is okay for some absolute basics and the rest of the work goes to actual frontier models.
jimjimjim 2 days ago [-]
How long ago did you try it? Gemini models seem extremely capable to me. I've been using them in ag for over a year and to me the results are great. switching to opus in ag doesn't produce any better results for me
kranke155 2 days ago [-]
All the power users are always jumping around by now and many hold the smallest subscription with all the models or at least two.
Narrating this as if it’s the game of thrones is a fun exercise, but it’s just becoming commonplace now that models jump ahead of each other on a weekly to Monthly basis.
nomel 2 days ago [-]
> and many hold the smallest subscription with all the models or at least two
This was part of the issue, as stated in their comment. We couldn't jump around to try it for the reason that is wasn't made available to, as you say, "all the power users".
postalcoder 2 days ago [-]
You're not convincing me. The war for compute within google is well known by now. Every single product decision stems from that.
Do you really think people in google are happy about how long it took to release this model?
rebolek 2 days ago [-]
Why should I care?
iforgotmypasswo 2 days ago [-]
Why did people jump to Claude? I redirected company resources to OpenAI Pro 500 accounts and away from Anthropic for most use.
We’re all running on Astra medium the whole week and loving it.
I’m curious what I missed.
alex0015 2 days ago [-]
I'm still a codex man myself but lots of people feel like opus 5.5 just gets it for some projects in a way that astra doesn't. Especially given the fact that astra and sol 6.1 are often at capacity late at night during US hours, when lots of people want to run longer unsupervised projects while they sleep. There's a general sense I've felt reaching some comment threads that openai failed to react with something better than opus 5.5 and are rushing to release small tweaks to make people feel like they're still getting something new.
bitexploder 2 days ago [-]
Sol is just hilariously slow. Even Luna is slow. It is unusable compared to Claude. I was pretty unhappy with Opus 5 and really gave OpenAI a shot for personal projects, but their quotas are just too small and their capacity issues in the last few weeks have just been untenable. No good. Opus 5.5 is just a very good model, very efficient with usage, gets things done at a reasonable speed. Sonnet 5.5 is also a very nice worker improvement. It is probably on par with Opus 4.6-4.8.
iforgotmypasswo 2 days ago [-]
The sense I get from your comment and the GP comment is that OpenAI’s pricing changes alienated individual users for the sake of the enterprise market. I live in the enterprise market, and was quite pleased with the changes. We don’t really use Sol 6.1, just Astra.
$500/month for 40+ hours a week of Astra usage, with ultrafast mode available when you’re near the weekly reset has made our jobs so much easier. Our product has some sciencey bits as well, ands Astra kills it as a rubber duck in this area.
As a consumer whose 20x account got downgraded, I am a bit sad. I can’t justify $500 a month on personal AI spend. As an engineering manager, I could not be happier with OpenAI’s services. Also, Codex App Server is amazing.
smoe 1 days ago [-]
I'm a bit confused by this comment. I thought Enterprise customers didn't have access to fixed price plans like the $500/month personal tier and instead paid per token? If that's available to businesses, Astra seems like a no-brainer. But at least the regular Business plans only show me the $25 and $125/month options and the latter is a very recent addition.
magimas 2 days ago [-]
> Our product has some sciencey bits as well, ands Astra kills it as a rubber duck in this area.
In my experience, this has always been the advantage of OpenAIs models.
Throughout iterations, Claude is better at pure coding, but the GPT models are better at "scientific reasoning".
discreteevent 2 days ago [-]
> Also, Codex App Server is amazing.
Hey, account created 39 days ago. Did you really have to tag that on after all the other stuff? Could you make it any more obvious?
That said this thread is hilarious because it's just an open ai sales manual debating with an anthropic sales manual. It's funny, but it's not, because the truth is that you people are just shitting all over the commons. Someone needs to kick you out but that's not going to happen here.
iforgotmypasswo 2 days ago [-]
This product has made my life substantially easier. I have no affiliation with OpenAI.
Claude refuses to help you build your own harness off a subscription, and OpenAI actively encourages it with Codex App Server. So you’re just in an AI IDE customized to your job and specific needs ALL THE TIME. You know that good feeling you got when your CI pipeline was finally working, back when that involved speed reading documentation and guessing, or staying at work until 10PM? You just get that all the time because life is constantly improving. And your tools are beautiful and riddled with Easter eggs you put in them. There’s so much more time to test and talk to stakeholders because you are able to distribute the thing you are actually good at -solving people’s difficult problems- while following best practices. You used to have to weigh those two every day and it was terrible. And everything was always late. You had to cut so many corners you were delivering circles.
Now life is fun and I’m over-delivering. Constantly. With super high code quality, better test coverage, easier audit prep, and I just explain what I want (technically) at a high level to AI. I still have to code review, and it sucks. Not because the AI needs it, but because auditors need it. I think I can make it faster and easier in the near future with another custom tool.
I actually have time to make architecture diagrams. I have never had enough time to build that level of documentation.
With Fable, I felt the promise of this coming. With Astra, I feel like we finally had the resources to fully professionalize and supply our software team.
We actually need to hire more people next quarter.
I am with the creator of Rails on this. It is a time of joy. Now is the time to become more ambitious.
bitexploder 2 days ago [-]
:shrug: I spend a lot of those token make local models better, resurrecting old hardware to make it more accessible and usable. Frontier models and AI is cool. This is my lived experience doing real systems engineering projects. My real sales manual is: https://stealthgoat.com/deepswe-flash-next.html local models are kind of awesome now and everyone should be using them in addition to frontier models. I will build the stuff I want which includes optimizing local inference engines until they are at my hardware ceiling. I use this capacity in my house to do reversing, security research, and internal OpenClaw type stuff I find useful.
holoduke 2 days ago [-]
They are all very slow at the moment. Some tasks are taking hours to even days. These are DevOps related tasks in which I create hundreds of servers in an agentic way. It works. But it's slow. That's still annoying.
Al-Khwarizmi 2 days ago [-]
Just for the record, I'm a Spanish user with Google AI pro (€20 per month or so) and my app has Argon now. (Yay!)
julianz 2 days ago [-]
What about Antigravity? I don't get Argon 4 in that.
topspin 2 days ago [-]
"model was unavailable for anyone to try"
Oh yeah. Completely forgot about that. Big headlines and big claims, but no one could actually use it and report on it.
Does seem like they don't quite grasp how the hype train operates. Or maybe they don't care as much.
msabalau 2 days ago [-]
Gemini 4 is announced be available to Pro users in the Gemini app.
I mean, I heard rumors too that it would be ultra only, but that doesn't seem to be the case, unless you are specifically talking about antigravity?
Realistically, market adoption isn't going to be the result of terminally online people (like me, I confess) following things way too closely on social.
doodlesdev 2 days ago [-]
There weren't "rumors" that it would be Ultra only, that's what Google has announced two weeks ago. Happy to see they've went back on that, but I haven't seen Gemini 4 on any apps till now, and I'm a Google AI Pro subscriber.
edit: just a few hours after this comment I got access to Gemini 4 on the Gemini app. Yayy! :)
Ternari 2 days ago [-]
They never said it would be Ultra only. They said, "starting with paid API customers and Google AI Ultra subscribers" in the initial announcement.
That's correct, I've misquoted it. Although, if I understood correctly, it got released to all plans together today right? Or did AI Ultra subscribers have access to it since the first announcement?
mrybczyn 2 days ago [-]
am on pro. updated my apps. no g4. sadge google.
reportingsjr 2 days ago [-]
I got it within the last hour, it is still rolling out.
topspin 2 days ago [-]
On pro as well. I have "4 Argon" in my Gemini web app model dropdown.
motoboi 2 days ago [-]
> At the same time, OpenAI had a disastrous dev day that led a lot of users to jump ship to Claude.
Can you elaborate on this?
simonw 2 days ago [-]
There was some backlash around the pricing changes - the $200/month tier offers less usage now, at the same time as the launch of the $500/month tier.
Aside from that I'm not sure what was "disastrous" - maybe the lack of a new model release to compete with the just-released Claude Opus 5.5?
keyle 2 days ago [-]
I went from using GPT exclusively to cancelling all my subscriptions with OpenAI.
'backlash' is underwhelming. They single handedly murdered 5.3 codex which was great by making it slooow and duuumb, and pushed people over to their new ones. 5.6 was a decent candidate but it was so slooooow.
And then double or triple the cost for Plus and Pro by nerfing their effective power? I never looked for an alternative faster. They really screwed the pooch.
Been quite happy with deepseek v4.1 flash, muse 1.3 and haiku 5.5.
AspireOne 2 days ago [-]
"Disastrous" is obviously a loaded term, and it can be debated, but from what I collect, the reasons are
- Dots. People don't understand it. Their presentation didn't explain it properly at all. Additionally, many people thought it's stupid, and that the cutesy mascots are dystopian.
- Voice mode didn't work in their presentation. The demo failed.
- 200$ plan goes 20x -> 10x usage. 500$ plan introduced, 25x usage. The contrast is stark.
- "Ultrafast": 8x pricing on subscription plan (6x API). That's 80$ input, 400$ output, 100$ cache write. Hello?! Additionally, since it generates 8x faster, it consumes your quota 8x faster. Per wall-clock minute, it's 64x more usage spend per minute of generating. People report that on the 500$ plan, their weekly usage is gone in 40-100 minutes.
- People believe gpt-6.1-sol was planned to be released as astra-minor, but OpenAI renamed it at the last possible time, because gpt-6-sol is a disaster (this I can get behind)
baq 2 days ago [-]
Pulled astra 6.1 at the last moment (allegedly) and only released a dirty looper sol 6.1. Also released dots which are unavailable in EU. There may have been more but no one cared.
nickthegreek 2 days ago [-]
and dots are only on $100+ plan.
baq 2 days ago [-]
Unpopular opinion: em-dash is fine
dsco 2 days ago [-]
I added Gemini 4 Argon to our video creation benchmarks.
The results are a bit disappointing given Googles scale and training data.
People should appreciate Google more for how all of their models have:
- Natively multi-modality
- Enormous context windows
- Reliably cheap prices
It's like they don't release a model until it passes the above checklist.
Most providers have some mixture of these three occasionally, but with Google, every model is consistent. This kind of consistency is really nice to have.
godbox 2 days ago [-]
And the most reliable. A year and a half ago this was the exact reason why I switched to Gemini API from OpenAI. OAI would make me wait up to 10 minutes for the first token in the most egregious cases. Gemini served me almost immediately every time.
sunaookami 2 days ago [-]
Huh? The constant failures in the Gemini API and overnight changes are well documented, there was a time from January to roughly July (maybe it still persists) where the API constantly spew 503 errors. This was so bad that I switched to OpenAI API instead. See e.g. https://discuss.ai.google.dev/t/frequent-503-errors-service-...
UnfitFootprint 2 days ago [-]
Cannot say I’ve shared that experience. Requests 504ed 5/6 times during my short trials with curl
dudeinhawaii 2 days ago [-]
Good point. On that note, it may be a bug but I noticed Argon compacting well under the 1M max in Copilot (around 400k) -- when set to 1M context. That may be a Copilot thing but I've never seen that with other models running 1M context. Could be a day-1 bug.
Outside of that, behavior is quite good, a bit slow, and definitely different than both GPTs and Claudes -- which is appreciated.
beardbandit 2 days ago [-]
As a personal anecdote I highly recommend not using the Copilot harness. Even with the same model I found it produced mediocre results against the first class frontier model harnesses.
antupis 2 days ago [-]
For my own benchmarks copilot cli uses 1.5x more tokens than opencode.
saejox 2 days ago [-]
Consistently buggy you mean. 1 in 10 tasks get stuck in reasoning loops or error out.
sva_ 1 days ago [-]
So cheap that people who pay for pro don't get it as a coding agent in antigravity lol
logicallee 2 days ago [-]
>Reliably cheap prices
do you have to pay those cheap prices eventually or can you just accrue an unlimited amount of overdue charges?
bergheim 2 days ago [-]
[flagged]
thehappypm 2 days ago [-]
They also are big contributors to open weight
bitpush 2 days ago [-]
What's a good LLM model not made by a billionare?
2 days ago [-]
mridulmalpani 2 days ago [-]
Gemini 4 Argon is a new leader in Pareto-frontier. Looks like, Google is back in the race.
For almost same intelligence (+15.56%, +15.18%), drastic difference in cost/task ($3.51 vs $0.41)
ianm218 2 days ago [-]
Why do these benchmarks often show models running on Max i.e. Opus 5.5 (max) is on the frontier? I would think it is never really economical to run max
jonas21 2 days ago [-]
By definition, the model with the best performance will be on the pareto frontier (regardless of cost).
seizethecheese 2 days ago [-]
No. Pareto frontier is cost performance tradeoff.
cloudbonsai 2 days ago [-]
What jonas21 is stating is correct. If you have the absolute best performance, you are automatically on the Pareto frontier because you’re the most cost effective one (or the only model) on that performance tier.
tibbar 2 days ago [-]
It is true that the model with the overall best performance is on the Pareto frontier, but it is not necessarily true that a given model's best cost-performance tradeoff is at max reasoning. (IE you could have a model that is on the frontier at medium reasoning, but at max reasoning becomes more expensive and not sufficiently good enough to remain on the frontier.)
wordpad 2 days ago [-]
If you're the best then you'd be thr first regarding of cost since there is nothing to compare to for that performance.
dfdydx 2 days ago [-]
As other have stated, your point is relevant only for the single best performance point. The question is for interior points of the frontier where the tradeoff actually exists. For instance why is GPT 6 Luna (Max)on the frontier rather than GPT 6 Luna (Medium)? It is not because GPT 6 Luna (Max) is the most performant model regardless of cost.
seizethecheese 2 days ago [-]
Huh, I guess you're right, but I interpreted jonas21's comment in relation to the above comment, which was about all models running on Max (which implies this would be a comment relating to all points on the pareto curve).
serioussecurity 1 days ago [-]
Yikes.
casey2 2 days ago [-]
Why is this downvoted? From my understanding dominated solutions aren't on the paerto frontier e.g. in this case where argon and opus 5.5 are tied the lower cost argon should be the end of the curve.
From the graph it's a flat line, but looking at the point yields a negligible increase with no error bars (indeed if you look at fable 5.1 it's measured as lower than opus and argon). This just seems like bad data or blatant favoritism at worse.
scotty79 1 days ago [-]
Check out slopalytics. You can draw a pareto line there with max reasoning excluded.
cheema33 2 days ago [-]
> Gemini 4 Argon is a new leader in Pareto-frontier. Looks like, Google is back in the race.
For chat, yes. But switch to code on the same page, and the picture changes. OpenAI takes the lead.
Silagi 1 days ago [-]
Further, this benchmark looks to have done too poor a job of canvasing the model/reasoning level space to claim to represent the pareto frontier. Only running 6.1 sol and opus 5.5 on max seems absurd given that their other reasoning levels represent almost the entirety of the pareto frontier above luna for coding.
vessenes 2 days ago [-]
Muse reminded me we should have a time based chart as well - this year I hope we will begin to see time to completion stats at various intelligence tiers. The waiting makes a lot of things more tiring.
BrokenCogs 2 days ago [-]
Every release I see a new benchmark with a new frontier. These graphs are just meaningless now
abeppu 2 days ago [-]
In particular, where cost is the x axis, a well-funded company with a good but not best model can discount their new model until it reaches the frontier. In some sense this is as much a statement about the efficiency of markets in the presence of public measurement-based comparison and public list pricing as it is about research pace.
kridsdale1 2 days ago [-]
While true, Google genuinely has invested the most in computer and energy efficiency for LLMs.
LPisGood 2 days ago [-]
This makes sense, but what evidence is there that this is true?
bootsmann 1 days ago [-]
They run an LLM to summarize every search query for free and still turn more profit than ever?
LPisGood 1 days ago [-]
That’s why it makes sense. It’s not evidence though. I was looking for investment numbers, papers, blog posts, etc.
yipinwong 2 days ago [-]
you can give it a meaning if you see the relavivity between models. no need to disregard the stat at all.
2 days ago [-]
jjcm 2 days ago [-]
Image->html test for Argon. I'd put it at B tier, just under gpt 6.1 sol. It isn't A tier like Opus 5.5 or Astra.
Nice research, you are right! Seems that low power mode was on. I have been using the web, Netflix, Apple TV app, and anything else just fine without ever noticing a thing though.
So that all being said, a normal website should work totally fine on low power mode, and this should definitely be a negative on the Opus implementation.
jjcm 2 days ago [-]
> and this should definitely be a negative on the Opus implementation.
I would disagree with this based on two things:
1. the prompt, which explicitly states to render the planets in the background in 3js (thus the gpu implementation).
2. the purpose of this, which is a test of its ability to convert an image design into a html/js/css implementation that matches it as closely as possible. I've talked about this before when it's come up[†]. The purpose is to be a good test, not a good design or website.
For reference, here's the full prompt used:
> Create a web page implementation from the following instructions: https://diffui.ai/build/Futuristic_Interior_UI_build.md?auth... dont reference any other folders that have similar builds. Focus on the flowing SVGs. Animate elements that make sense to be animated. For the starships that have planets in the background, ask for high quality images with the windows as a transparent background, and recreate the planets behind them in 3js. It should be one single page application that I can just open the index.html file to view.
Even after ensuring GPU acceleration is enabled on Firefox on Linux, it's still sluggish.
daemonologist 2 days ago [-]
The Opus 5.5 implementation is noticeably slow on my machine as well (Firefox, Fedora, Gnome Wayland, AMD 6800U).
jjcm 2 days ago [-]
Interesting! Do you have webgl disabled for incognito or for chrome in general? I wonder if it's trying to do software rendering.
FWIW, I have literally the exact same configuration (M3 air, google chrome). Here's a recording on mine where I'm seeing 60fps: https://non.io/video/video-3587018401.mp4
Gallows4574 2 days ago [-]
Mine doesn't stutter on on Firefox on M1 Max, but it pins 7 cores the entire time the page is open, not just while loading. It also uses 50% of GPU in addition to that.
Edit: The argon built site pins 1.3 cpu cores. The Sol 6.1 site uses 0.18 cores.
dr_kiszonka 2 days ago [-]
Unless it is something with the playback on my end, your animations and scrolling are a bit jerky too.
ziiinq 2 days ago [-]
[dead]
MaxikCZ 2 days ago [-]
Mac Air M1 in Safari here, buttery smooth.
wireless6948 2 days ago [-]
Also stuttery on Edge (Chromium), so there might be a pattern here
niek_pas 2 days ago [-]
Fine on arc search (which I think is just WebKit) on iOS.
hedgehog 2 days ago [-]
Maybe a Chrome bug? It's fine in Safari on an M1.
piperswe 2 days ago [-]
Also horrifically sluggish on Firefox on Linux.
hedgehog 1 days ago [-]
Weird. Hopefully they converge, I don't miss the days of wildly divergent browser engine behavior.
That's really interesting I think- literally putting the layout into ascii language to reason about.
sergiotapia 2 days ago [-]
Damn opus 5.5 is just such a good model - what kind of moloch sacrifices are they doing in there
capnjngl 2 days ago [-]
Truly S-tier. It's sharp, creative, intuitive. Can't believe how much better it is than Opus 5, then they went ahead and made it cheaper and faster too.
jjcm 2 days ago [-]
I'm constantly blown away by it. I do think Astra is slightly more intelligent (although worse at this particular benchmark), but Opus 5.5 has become my default model for 99% of what I do. It's so good.
danpalmer 2 days ago [-]
Very subjective, but I'd say the Argon one is the best here. The other two don't quite do LCARS correctly, there are a bunch of lines/curves that are the wrong angle, don't line up, etc. The Argon one looks to me the most like a designer would have done it. Of course this is up to personal taste to some extent.
jatins 2 days ago [-]
Are you checking on desktop? On phone Argon one is completely broken
awkwardpotato 2 days ago [-]
Argon's build is pretty much unusable on mobile, the header takes up half the screen
Opus one has by far the worst performance, the only one that's stuttering and making my laptop fans very noisy.
Panzer04 2 days ago [-]
Opus is way slower in firefox on my machine than the other two.
rrr_oh_man 2 days ago [-]
Off topic, but I want to customize a starship now
rrr_oh_man 1 days ago [-]
Oh, fuck, I just realized those are actual apps?! Holy cow.
demibabs 2 days ago [-]
B tier? For me the argon one is really quite bad on mobile; the nav is messed up enough to make the page kinda unusable.
derac 2 days ago [-]
On firefox on linux, argon looks great and the other two have graphical issues.
vessenes 2 days ago [-]
That Argon build is really bad on mobile fyi
civvv 2 days ago [-]
Jesus Christ, so much "stuff" crammed into so little space. What a nightmare of a website.
jbrooks84 2 days ago [-]
How on earth can you say one is better than the other? It is subjective.
jjcm 2 days ago [-]
For an implementation test, there are a bunch of very objective things you can measure.
IE for Argon, the third section doesn't have a top view of the ship with labels going to each section like what's in the design. Opus has it, and animated it as well. One failed to do as instructed, the other went an extra mile.
scirob 2 days ago [-]
Google continues to be the best in German language and Gemini 4 is a significant jump in our benchmarks
It's a great model. I have had it trying to decrypt the Voynich manuscript for the past couple of days without being biased by other research/other people's attempts. If anyone is interested: https://github.com/d2kx/voynich-research
kzrdude 2 days ago [-]
How ironic that the Readme is completely unreadable, like gibberish rivalled only by the Voynich manuscript itself.
But id the progress of decoding it is legit, you should write something properly about it!
yipinwong 2 days ago [-]
I am still lost as I am not linguistically trained well (jargon-free didn't help for me).
What kind of progress was made with Argon with the Voynich?
jdthedisciple 2 days ago [-]
Have u tried this with GPT Astra as well?
shawabawa3 2 days ago [-]
Wow I actually tried this with fable a few weeks ago and got nowhere
It looks like you've managed to decode it? Is that legit?
shawabawa3 2 days ago [-]
[flagged]
babelfish 2 days ago [-]
I saw 'jargon-free' and my eyes immediately glazed over
d2kx 2 days ago [-]
hahaha. I also build a site (also using Gemini 4 Argon) that makes the data more accessible: https://www.dmh.dev/voynich
mattmanser 2 days ago [-]
> An interactive companion to our ongoing exploration
Eyes glazed over again.
Did you even read your own site before you posted it?
One of the first pages I opened:
> Smoking-gun proof of plant-label ingestion: in Row 2, the central cut root is labeled okol (f88r.15), and directly underneath it on Line 2 (f88r.19) the recipe writes...chokol daiin qoekol qoekol qockhol okol cheol...!
It's just gibberish.
swingboy 2 days ago [-]
Well, the manuscript _is_ written in an unknown alphabet, hence the “chokol daiin…”
mattmanser 2 days ago [-]
It's the English bit that I'm referring to as gibberish.
It only makes sense in a wider Claude session, and even then it's like he's got Claude set to super-boot licker mode.
case540 2 days ago [-]
How about you post something you've done/built instead of just insulting other people. Spend your mornings using people stuff just to poke holes in it?
HenryMulligan 2 days ago [-]
Don't judge, he/she might work as a Test Engineer!
Macuyiko 2 days ago [-]
Adding my five cents: initial usage through OpenRouter has been great. Very fun model to spar with, clever, intelligent. On par with Opus 5.5 but without the Claudism's and more of a Sol type of conversation. Though speaking of Sol, I've been very dissapointed with Sol 6.1 to the extent that I feel OpenAI has neutered it compared to a few months ago. Especially when giving it goals it will happily run in dumb circles just burning tokens without any real work.
scrollop 2 days ago [-]
Here are sites with ongoing measurements checking if a model has been nerfed:
Yes sorry, I meant compared to the flagship models of a few months ago, < 6.1.
koyote 2 days ago [-]
Definitely agree. Sol 5.6 was great when it was released and for nearly a month.
Since then they cut the price but the intelligence (of 6 and 6.1) has gone down to Sonnet 4 levels. I am back in a world where the model is lazy, cuts corners, ignores existing coding standards and context, makes stupid mistakes etc.
I am not sure how those models still rank so high in benchmarks when real-life usage is so obviously bad.
DuncanCoffee 1 days ago [-]
They removed the 3.1 pro model from their offer, and version 4 is locked to ultra customers, so basically pro customers are paying for gemini 3 flash.
State of affairs [1]:
> My whole project and career depended on the 3.1 pro workflow. Please, bring it back
I ran it through my own coding eval today using pi/OpenRouter (11 small bug fixes from my own projects). It solved 10 of 11. Compared to Sonnet 5.5 and GPT-6.1 Sol, a median run cost me $0.52 vs $0.18 for Sonnet and $0.09 for Sol, and took ~4 minutes vs 32 seconds and ~1.5 minutes.
It thinks a ton, ~24k output tokens a run vs ~3k for Sonnet. It missed a solution by thinking its way into a 10 min timeoout.
Something to look out for - if you're using pi, it doesn't have Argon in its catalog yet and reports cost at Kimi's rates, about a third of what OpenRouter actually bills.
freakynit 2 days ago [-]
"Argon autonomously turns raw video to a physics-accurate simulation, estimating depth and physical boundaries, and writing the exact simulation code required to replicate those real-world environments."
This is crazy if it can work accurately.
ArashEdalat 2 days ago [-]
Geometry could be understood today with the model but what also really matters is the sensing, friction, mass how smooth the subjects are, dexterity is a very difficult problem so solve, not sure how simulations could handle such factors
verdverm 2 days ago [-]
Game engines have many of these features. At one point, iirc, Unity was positioning to be the simulation environment for world model agents before we spoke about it in these words.
kridsdale1 2 days ago [-]
Every animal can do this, in order to run and jump and catch.
34679 2 days ago [-]
And because of technology like this, soon every robot will be able to as well.
> If it's on HN and is a meme at this point, it will end up in the training set.
> It's kind of fun to imagine that there is an intern in every AI company furiously trying to get nice looking svg pelicans on bicycles.
Google listened and assigned a few interns, I guess, for the task
JoshTriplett 2 days ago [-]
That's exactly what I was thinking when I saw https://www.assbench.com/ posted a while back: "oh no, this is going to be somebody's job".
rubyn00bie 2 days ago [-]
NGL, I think this might be my favorite benchmark, ha. While the Chinese labs asses are a bit off, I certainly have big hopes for their next versions. Their hearts are certainly in the right place.
gedy 2 days ago [-]
It's the smaller training data /s
codethief 2 days ago [-]
Unfortunately, for me the renderings crash or hang in both Firefox for Android and Chromium/Vanadium on a Pixel 10 Pro. :\
lp92 1 days ago [-]
Works fine on Android + Firefox for me.
ngruhn 2 days ago [-]
omg that's hilarious
NoDodgeQuestion 2 days ago [-]
prompt is to detailed (llm written?) better test of model creativity is to not constrain prompt
eli 2 days ago [-]
SVG is useful for actual tasks, not just looking good in a silly "benchmark"
CuriouslyC 2 days ago [-]
Beyond SVG, these things have been RL'd on 3d reconstruction (which carries over), and I expect given the public reaction to recent frontier advancements in 3d capabilities that this will get allocated a larger share of the post training budget in the future.
PorciiVorbesc 2 days ago [-]
What I'd kill for would be an app where you take a pic of scribbles and drawings on a whiteboard/notebook and the app turns it into SVGs you can then continue to manipulate digitally on PC/tablet.
gh0stcat 2 days ago [-]
Why would you kill for this? Why not just generate svgs with any editing software?
PorciiVorbesc 2 days ago [-]
>Why would you kill for this?
Figure of speech.
>Why not just generate svgs with any editing software?
They can represent something in vector, but it's always very hard to use the result for anything else. Like, the paths doesn't make sense, aren't grouped to the logic elements etc.
eli 2 days ago [-]
ChatGPT does that decently well
PorciiVorbesc 2 days ago [-]
Last time I tried, it did not. That's why there's a place on the market for an app just tuned to handle the transition seamlessly, rather than having you fucking around with prompts in LLMs.
TeMPOraL 2 days ago [-]
I for one don't want a whole app for that, not anymore. It's a small task whose result is an input to something else - ideal for a tool for LLM (or script) to use.
PorciiVorbesc 2 days ago [-]
>I for one don't want a whole app for that
Everyone says that until ms, apple, google, etc introduces this polished feature as part of their tablet imaging editing/document collaboration office suite. Then everyone can;t get enough of it.
> It's a small task whose result is an input to something else - ideal for a tool for LLM (or script)
Have you actually tried this? If it's a small task then why can't LLMs actually do it? Kind of like the pelican test. Every LLM can do a SVG pelican but most of the results are unusable in practice for production/productivity purposes.
Every LLm can extract some half-broken SVG hallucination of your jots, but I'm asking for an app that's polished and works accurately 99% of the time, which means a dedicated app/model tuned specifically for this task, and nobody makes this yet.
TeMPOraL 2 days ago [-]
> Everyone says that until ms, apple, google, etc introduces this polished feature as part of their tablet imaging editing/document collaboration office suite. Then everyone can;t get enough of it.
I don't feel like I'm a representative sample of general userbase, so I won't dispute this too hard.
For me, it's rare but it does happen. Still, 90% of my mobile experience is "how is this so broken, why is it missing the obvious follow-up feature that completes a workflow, and why, again, am I not using a laptop for this?".
> If it's a small task then why can't LLMs actually do it?
What if they actually can?
This is exactly what Karpathy was talking about the other day[0]. If your only LLM experience is the default chat app in chat mode, then this doesn't make sense. If your experience includes using agentic harness like Claude Code/Cowork / Codex/ChatGPT Work, then it's obvious.
The LLM app in "chat mode" will just zero-shot you a pelican, or digitize your notes into SVG, and maybe improve on it a bit if you complain about the result. The same app[1] in "agentic mode" will digitize your notes, refine the SVG against the original image, reduce it to probable strokes based on high-frequency patterns in the photo and knowledge of how people write, OCR it, save both the text and SVG to your library, then extract parts you wanted and insert them into a document, typeset the rest of it, and print it on your printer - all in response to a single prompt.
LLMs in agentic mode are very good at stringing long and advanced workflows; today, if you give them shell access or computer use, they will use regular apps for you if that's the easiest way to accomplish the goal.
> Have you actually tried this?
Yes![2].
That's why I can confidently say: there is no limit to this pattern. LLMs can handle apps just fine; giving them tools just saves time and tokens.
Another example: I'm currently resurrecting a 30 year old game, and the other day I asked Claude to hack on a way to run it headlessly, so I can use my computer while it plays with it. It did, and then immediately leveraged it to just causally run several instances of the game in parallel, entirely in the background, whenever it needed to playtest something.
[1] - Claude and ChatGPT now have both modes in their mobile, web and desktop apps. I personally use Claude Code a lot, but exclusively through these surfaces (I can't be arsed to deal with TUI at home anymore). IMO, these tools are already within reach of regular users - the users just didn't notice it yet.
[2] - Not with SVGs, but equivalent with voice recordings, and yes, Claude Code set up and is managing the whole local processing pipeline. Recordings go in, Markdown (or more advanced signal processing) comes out, and in itself, it's not a task, but merely a step.
matsemann 2 days ago [-]
That's a very long-winded way of saying no, you haven't tried it. The point of converting something to svg is to be able to manipulate the elements, not just having an svg that mimics pixel perfect something. That's been possible for ages before llms. But the result is hardly usable if it's made up of random lines and arcs, not how someone would actually have drawn it.
TeMPOraL 1 days ago [-]
Have you seen a typical SVG? No one except maybe LLMs make those semantically. It's usually just whatever InDesign or Inkscape vomited out, given construction that's about as semantic as formatting in a typical Word document.
matsemann 1 days ago [-]
Can you please slow down and read what's being written? Didn't even mention semantics.
One fictional example of what I mean: lets say I draw someone holding a purse. That would be a shape of their torso, with the purse overlaid. I could move those independently, and moving the purse would show the torso underneath.
However, using a png to svg, it would often just outline everything. So it's not a torso shape with a purse in front. It's a purse with a torso cut out around it, maybe even merged together as lots of individual shapes. If I were to delete the shapes making up the purse, it could be quite random what was left. Maybe some of the shapes did a double duty. Maybe it's a thousand small squares to make the purse. Maybe there is just a hole behind the purse etc.
So in the latter case, the svg isn't made up of logical elements, it's just made up of shape after shape after shape to mimic the pixels. But it's not usable as vector.
slyrus 2 days ago [-]
Of course it would be nice if google actually added SVG support to google slides...
> How many pelican riding bicycle SVGs were there before this test existed? What if the training data is being polluted with all these wonky results...
But Simon has demonstrated before that asking for any animal doing something silly will still get similar results from the same model.
Do you think this might be a kind of subconscious origin of your idea? A second of a move you saw in your early life / teens stayed with you as an example of an absurd thing to draw?
simonw 2 days ago [-]
It's possible, but I think it's more likely that I was inspired by the TikZ unicorn thing, my love of pelicans (we have thousands of them in the harbor ten minutes from my house), and my knowledge that bicycles are difficult for humans to draw.
orbital-decay 2 days ago [-]
It's still clearly not a part of any model's training set in the sense of pelicanmaxxing in particular. It's either a capability to generate arbitrary SVGs (or even something broader), or no specific training at all.
nomel 2 days ago [-]
Yet, they're all almost exactly the same in style and composition.
orbital-decay 2 days ago [-]
Certainly not because all labs are pelicanmaxxing on the same exact image. Most patterns are similar everywhere. That's the result of training on the same datasets and scraped web contaminated with other models' outputs, combined with similar training methods and resulting mode collapse. Try if e.g. dolphins knitting socks in spacesuits are similar on all/most models.
nomel 2 days ago [-]
> It's either a capability to generate arbitrary SVGs (or even something broader), or no specific training at all.
> That's the result of training on the same datasets and scraped web contaminated with other models' outputs
Maybe I'm misinterpreting though, or just semantics. I would consider contamination, or whatever the first example was that they all matched, as specific training data, along with all the duplications of. I doubt it's model collapse to the same pelican, without the same original example.
orbital-decay 1 days ago [-]
Perhaps you're right, this would probably take some rigorous forensics to resolve definitely. Somehow all models have very similar patterns in text, seemingly at random. I don't think everyone is trying to train on specific patterns on purpose, it's a pretty general phenomenon. (I'm talking about mode collapse, not model)
nomel 2 days ago [-]
This and your original comment aren't compatible.
orbital-decay 2 days ago [-]
How so?
afavour 2 days ago [-]
Certainly true but I think the question is “does targeting the SVG pelican mean that the quality of all SVG output improves?” If so it’s a worthwhile thing to target.
compiler-guy 2 days ago [-]
It's a little bit like any benchmark, especially microbenchmarks. It is useful as far as it reflects the broader use cases, but the risk of overfitting to the specific task and not the broader ones is very high.
They said they are training for svg not pelican svg specifically
mlmonkey 2 days ago [-]
Sadly, this is all that managers know to do: take a metric and try to optimize the heck out of it. They don't care about research, technology, "don't be evil", etc. And Google is run by managers now.
Expect Google models to be benchmaxxed as much as possible, but don't expect commensurate performance on real-world tasks.
anthonyrstevens 2 days ago [-]
>> this is all that managers know to do: take a metric and try to optimize the heck out of it. They don't care about research, technology
This is not true. It is needless negative hyperbole that IMHO doesn't belong on HN.
mlmonkey 2 days ago [-]
I'm speaking from 30 years experience in the Valley, having worked at several big companies.
agar 2 days ago [-]
While this might be overly cynical, it's also a powerful validation of the exact feedback loop that you want to have in an organization.
If what you say is true, then the next set of benchmarks will be written to illuminate the true reflection of real-world tasks. Then those get benchmaxxed so managers can show "numbers go up."
Repeat enough times, and you get materially and genuinely improved products.
The burden is on the press, outside researchers, and general tech industry to both write and care about the metrics that matter.
solatic 2 days ago [-]
There's just one problem - culture eats strategy for breakfast. You cannot govern culture or improve culture with metrics.
In large companies (Fortune 1000 etc.), you don't have much of a choice, the culture was eroded long ago, and corporate governance by metrics is a lesser of evils. But thankfully, not all companies are there yet.
kalcode 2 days ago [-]
For sure, they saw your comment and got some interns on it.
gcanyon 2 days ago [-]
Maybe so, and yet it's still not enough -- the pelicans all have flaws.
They look remarkably good, sure -- but not perfect.
sulam 2 days ago [-]
I end up getting SVG output for all kinds of things, and never once have I asked for, or gotten, a pelican.
fipar 2 days ago [-]
I have a silly pet "project" that I've been using with different models, where I provide an image of a patch sheet for the Behringer Poly-D synth and ask for an html version and this is the first time I get something that, while still needing some tweaks, basically looks like the patch sheet. Before this, the closest had been Fable, but that still was nowhere near the actual patch sheet.
Recently, I’ve discovered that Claude can render great SVG versions of user interface screenshots. I ask it to substitute simple SVG friendly landscapes for any photographic images that happen to be part of the screenshots. The results are much better than they should be, especially with Opus. Usually good enough to be used on a website as a way to get an infinitely scalable, much smaller version of a screenshot. But clearly, I should try the Gemini models on this real world task. That’s where we’ll find out if all of this training to beat the benchmark actually pays off.
Is it me or is there maybe some carryover happening where the pelicans are portrayed as expending different degrees of effort too? Like, the high effort pelican has a determined look on his face.
Is the prompt context telling the models what their effort level is, and the ambient presence of that concept of ‘high effort’ in the context window is leaking through and affecting the portrayal?
simonw 2 days ago [-]
I don't know if this is how the OpenAI effort levels work, but for Qwen 3.8 27B the effort level really does get translated into the prompt.
xhigh: "Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer."
low: "Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration."
rich_sasha 2 days ago [-]
As a cyclist, I tend to pay more attention to the bike shape than most people. Everyone knows what a bike looks like, but as with LLMs, most people would struggle to draw one - correctly connecting all the different tubes at correct angles etc.
I note that even these bikes are not actually rideable. For a bike to be stable, the front wheel axle needs to extend forward a little ahead of the fork - the fork must ”bend forwards”. This is what actually keeps the bike upright, not gyroscopic effects or some such. Given LLMs are generally good at basic maths, im surprised they get this wrong.
Maybe a bit less so in this case though. As per the announcements, the model was actually trained for this prompt, so was fed some training data. I can well imagine the data there was just not so good.
kevmo314 2 days ago [-]
That's absolutely not true. Zero rake forks are totally fine and very common on BMX bikes for barspins.
People drawing bikes, scroll down to see they having made renders of all the drawings. Hilarious
Bedon292 2 days ago [-]
I feel like perhaps this is a bit too nuanced of a critique. While I do see what you are saying on many images of bicycles, there are also many that that I do not see it on. Close to half of them, in a quick Google image search, have forks appear straight and the axle is dead center on the ends of them. If there is something there, I am not seeing it even when trying to specifically look for it.
I'll also note that, using a piece of paper as a straight edge against my screen, there is a very slight forward curve on the low and a bit more on the high responses. Medium does appear to be perfectly straight.
matsemann 2 days ago [-]
There are two things involved. What you are mentioning is the fork offset. Whether the wheel is mounted ahead of the steering axis. Either by the fork having a bend, or the mounting point of the wheel not being in the dead center of the fork. Note that this is measured in millimeters, and often not that visible in a picture. I'd say almost every bike except bmxes have it, but maybe not.
But even with zero offset, there is another thing in play. The steering axis is at an angle, a fork never points straight down. So even with zero offset, you still get a trail, which is the difference between the contact point of the wheel at the ground, and where the steering axis meets the ground. This trail is really what matters in the end, not how you get there.
Whether having zero offset or not isn't what matter in the end, but the trail. And you can achieve that with little offset as long as the fork is angled.
(so the grandparent is a bit right in their critique, but not for the reason stated)
sokka_h2otribe 2 days ago [-]
The front axle is in front of the fork in that photo.
jonas21 2 days ago [-]
It looks exactly like the images the model generated. The fork certainly doesn't bend forward -- it's perfectly straight.
IsTom 2 days ago [-]
At the place where it splits in two it also goes shifts forwards a bit
adwn 2 days ago [-]
> Given LLMs are generally good at basic maths, im surprised they get this wrong.
1. Modeling and analyzing spinning, three-dimensional objects goes far beyond "basic maths".
2. The prompt was for an SVG of a realistically impossible situation (pelican riding a bicycle), which together already implies a stylized depiction. Why on Earth would you expect it to generate a 3D model of a bike, do a physics simulation on it, iterate until it satisfies certain contraints which are only hinted at in the prompt, and then project it down to a simplified 2D cartoon drawing? That's not how LLMs work, and that's not how we want them to work.
whoamii 2 days ago [-]
Simon’s headstone is going to have a pelican on a bicycle, isn’t it
mceachen 2 days ago [-]
He deserves a full parliament
gordon_freeman 2 days ago [-]
At some point Simon needs to move from Pelicans to something more complex. Every time a new model is launched by major Frontier company, I already know with certainty that first HN comment on the news would be about how Model is drawing the Pelicans.
tyre 2 days ago [-]
Agreed. It's a fun cultural tidbit but not really a useful benchmark for model capabilities.
ionwake 2 days ago [-]
did an LLM thats not very good at pelicans type this
saberience 1 days ago [-]
I stopped looking at Pelican pictures about a year ago when I realized they mean nothing.
philipwhiuk 2 days ago [-]
There's plenty of highly complex benchmarks for this stuff. Simon doesn't need to do that.
YZF 2 days ago [-]
Why are all these AI's making the pelican's pouch so large though? Unless they have some huge fish in there or something they don't tend to look like that... Is it that it's drawing a "cartoon" pelican or something?
Pouchbench. There's an RLVR grader that performs pouch detection and area estimation. Bigger pouches, bigger reward. Soon, all will be pouch.
saaaaaam 2 days ago [-]
A pelican with an empty pouch would have no need for a bicycle to get home.
CamperBob2 2 days ago [-]
And certainly not one with a basket.
ameliaquining 2 days ago [-]
Well, a photorealistic pelican would generally be in a raster-graphics format rather than an SVG, so yes, cartooniness is expected.
diseasedyak 2 days ago [-]
Pouchmaxxing
ASalazarMX 2 days ago [-]
The SVG is cartoonish, and to be honest, most of human-made SVG art is cartoonish or stylized. It's not a format with realistic expectations.
postalcoder 2 days ago [-]
It's obvious OAI hillclimbs the image generation -> SVG loop. I'd put money down on that being why OAI's pelicans have a yellowish tint that matches the yellow tint of OAI's earlier image models.
thefourthchime 2 days ago [-]
It does very good at Pac Man. I would say as good as Opus 5.5, it was slightly cheaper, but took twice as long.
For those who haven't seen it, Argon can use fourier circles to generate an animated 3d pelican -- I think the pelican is 3D and the motion was added by an inverse kinematics solver?
https://x.com/fofrAI/status/2110043023954284889
huseyinkeles 2 days ago [-]
Wow, those are I think the best looking pelicans ever so far
How many cicles before pelicans become photoreallistic SVGs and them come back to stylistic artistic abstract renderings?
Also, that first tweet contains a commentary that, for me, explain a lot about the public misconception about how current AI works:
> wait, does this mean you guys are actively training svg generation? i was mostly amazed by it thinking the models doing better at it by itself due to becoming better at reasoning.
No, it's not better reasoning that drives current AI, it's better examples we have been giving them. Whole army of people generating content for the frontier labs.
Also, reasoning improving the out of distribution part, like computer use, tool use etc.
TeMPOraL 2 days ago [-]
> public misconception about how current AI works
Another case where allowing oneself to anthropomorphise LLMs a little yields better intuitions.
People don't get more skilled by raw reasoning, either. They improve their skills with better textbooks (pre-training data), more practice exercises (RL data), and generally goofing around (self-play).
LarsDu88 2 days ago [-]
simonw, your pelicans are so famous now that the frontier labs can just RLVR for SVG images explicitly.
It's a simple manner of prompt...generate concept art with nanobanana, measure, llm creates svg with text, measure the distance between rasterized svg embedding to nanobanana art....do post-training RL on a 3 million such images and matching embeddings.
I swear this will be a canonical example of Goodhardt's Law in action.
You have to switch to a new benchmark that isn't so easy to run a verifiable rewards pipeline on...Perhaps a longer horizon task where it's easy to mess up midway.
jameshart 2 days ago [-]
Why? Don’t you think the steady improvement in pelican bicycling SVG art is itself a worthy field of endeavor for LLMs to be optimized for?
Yes, Simon has made pelicans a benchmark and now models have grown very good at rendering them. Is there a reason to believe that this isn’t manifesting as useful skill in general in SVG generation?
wasfgwp 2 days ago [-]
The sample is too small. As long as the models are reasonable close enough to each other there is really nothing you can learn from this benchmark when its just one prompt.
It would perhaps be slightly interesting if the prompt was run 100 times for each model. I doubt the pelicans would look identical.
2 days ago [-]
yuhmahp 2 days ago [-]
doesn't this mean that you should change the pelican test to something else?
simonw 2 days ago [-]
If a result is suspiciously good I run a spot check with one of my many other SVG things.
But there are definitely issues with your armadillo SVG?
1. the pedestrian traffic light's hand and "wait, citizen" text are not inside the light itself
2. The armadillo's nose is outside of the astronaut helmet
Isn't that just proof that the pelican has been benchmaxxed?
geniium 2 days ago [-]
It starts to be so good that the model doesn't really matter. Each model starts to have its own personality, and it's the personality of the drawing, like a real artist, that defines the style of the pelican, and that makes it harder and harder to decide if it's good or bad.
petra 2 days ago [-]
That's true.
I've noticed that chatgpt is working harder. But still I prefer Claude, it just feels nicer to me.
geniium 1 days ago [-]
I like to keep both at hand and i still like to switch every now and then between them
vessenes 2 days ago [-]
That pelican left leg stays so firmly on the left side of the bike. Superintelligence ftw
tintor 2 days ago [-]
Front bicycle wheel in all Argon examples is still wrong. This bicycle will not be stable. Axis of rotation of front wheel needs to be in-front of axis of rotation of steering wheel. Frontier VLMs still do not understand how bicycles work.
epihelix 2 days ago [-]
Check the "high" thinking output again. For what it's worth, it's there.
However, how the front fork rake affect stability is I believe a little complex. I doubt this old article is the last word on the subject, but it's a great read nonetheless, and note especially the ultra-stability of the reversed fork bike!
You gotta fix your tool on mobile browser, both firefox and chrome don't work.
gcanyon 2 days ago [-]
Excellent, but not perfect -- if google really is training to pelicans, then clearly they are not capable of perfect pelicans.
As one point, the high pelican's arm is see-through/messed up.
They do look worlds better than previously, to be fair.
NooneAtAll3 2 days ago [-]
> Google are the only lab who have publicly shared that they have specifically trained for SVG output
wasn't the whole point of pelicans is that labs *wouldn't* train for that?
so now the benchmark is ruined
bigcat12345678 2 days ago [-]
At least you should make sure this post is entirely generated by agent, otherwise I'll not grant my agent the credibility score on this comment.
w29UiIm2Xz 2 days ago [-]
Since we know jagged intelligence is a thing, how useful of an indicator is the pelican test, regardless of if labs are optimizing for it or not?
KeplerBoy 2 days ago [-]
That of course depends on what you want the model to do. If you are into artistic SVG rendering, it's still very relevant.
bilsbie 2 days ago [-]
Regarding the pelican test. Maybe someone will know this.
Modern models can make nearly photorealistic games in threejs. So why do they struggle with these pelicans?
simonw 2 days ago [-]
The models that are really good at 3D modeling tend to also be really good at pelicans. I think the skills are strongly related - it's about having good 2D (and now 3D) spatial awareness.
The test is always going to be hard though, because it's an absurd image to ask for.
> it's about having good 2D (and now 3D) spatial awareness.
this is a really profound insight
agar 2 days ago [-]
Models seem to have very interesting blind spots. While https://assbench.com/ may seem like a joke, the results are very illuminating. One might expect far better results given the detailed prompt, allowance for render checks and iteration, the number of human figures the models are trained on, and clarity of tool use (threejs).
However, result quality really drops off once you get out of the top 5 or so.
(NSFW note: all buttocks are covered by fairly conservative bikini bottoms and show no genitals, bulges, etc.)
jephs 2 days ago [-]
The pelicans are one-shot with not tools: Think a bunch, write a bunch of SVG, done.
The pretty 3d stuff you see are agents controlling a Blender MCP in a loop where they look at the scene, think, adjust the scene, repeat.
simonw 2 days ago [-]
Yeah, that's a good point. My basic pelican test doesn't allow the model to render their work and iterate on it. The 3D models all come from agent systems that are rendering with Blender or screenshotting with Playwright along the way.
agar 2 days ago [-]
Have you looked at assbench.com? Even with threeJS rendering, screenshotting with Playwright, and iteration (check the prompt) some of the models are shockingly bad. I wonder whether the rendering quality is affected by the requirement for skin physics on click; if so, it raises a valid question about models creating pretty pictures but break when interactivity is required.
(As I said in a separate comment above, results are all bikini-clad and not revealing. However, whether it is NSFW is up to you/your work policies.)
vineyardmike 2 days ago [-]
I’d really like to see what each model comes up with if given a basic harness to render and iterate. They’ve all gotten pretty good at basic one shot pelicans, but seeing what they do when given a direct goal, and even getting a time lapse of each image as they edit and adjust would be really fun.
AspireOne 2 days ago [-]
Very easy to do. It's a nice silly thing to try, so I'll conduct this very important experiment and update you on the results if I remember to do so
epihelix 2 days ago [-]
If you allow rendering and iteration on the basic pelican test, does it improve the final output?
simonw 2 days ago [-]
It didn't last time I tried that, but that was for GPT-5. I need to try again with more recent models.
TeMPOraL 2 days ago [-]
"Pelican on bicycle in SVG" is an adversarial example on multiple levels - out of distribution, ill choose of tool for the task, and anatomical impossibility of real pelicans using typical bycicles.
charcircuit 2 days ago [-]
But often these models mess up drawing even just the bicycle part.
duckmysick 2 days ago [-]
I remember a submission a few years ago about a guy asking others to sketch bikes. Most of the time the drawn bikes wouldn't be rideable.
A lot depends on what the models were specifically trained for using reinforcement learning. Modern models are generally OK and everything, but they're only really good at the things they have extensive RL for.
ben_w 2 days ago [-]
> Modern models can make nearly photorealistic games in threejs. So why do they struggle with these pelicans?
Are you sure they're not fetching free asset packs for that?
simonw 2 days ago [-]
Yes. I've been doing a bunch of 3D experiments with recent models, you can check the session logs and see if they've retrieved anything. They're building the 3D models directly in code.
I even built an environment that allows execution of Python code to render with Blender but allows no other interactions (including networking, I sandbox that off). They can create good 3D models just fine - here's a Faberge Egg by Gemini 4 Argon: https://tools.simonwillison.net/blender-viewer?url=https%3A%...
ben_w 2 days ago [-]
Oof (no pun intended), that's a huge step-change in competence over last time I tried this.
Thanks for doing all this research!
Perz1val 2 days ago [-]
Have you tried editing SVG by hand?
mroset 2 days ago [-]
The fish in the basket on high is a lovely touch.
travbrack 2 days ago [-]
The fish in the basket is a nice touch
Fraterkes 2 days ago [-]
Honestly surprising if other labs aren’t training for svg generation. Feel like SVG’s are going to be really useful for dynamic UI generation stuff that goes beyond cobbling Html together
2 days ago [-]
juleiie 2 days ago [-]
Nobody cares about the AI heckling pelicans.
Give me reverse engineering bench and agentic coding.
Leave making visuals to humans who actually have aesthetic sense unlike whatever made this mess
ionwake 2 days ago [-]
shun the heretic!
dyauspitr 2 days ago [-]
Holy shit, these are like art. Especially the bikes.
kart23 2 days ago [-]
still radial spokes on both wheels of the bike which is disappointing. no real bicycle is built this way
squidbeak 2 days ago [-]
Buddy, there's a Pelican riding it. No real cloud is like that, no real sea is like that, no real gulls are like that. And that's fine because it's a cartoon.
2 days ago [-]
kmod 2 days ago [-]
I've been dismissive of the SVG-of-pelicans benchmark, but last night ChatGPT (on the website, toggle set to GPT-6 High) generated some SVGs and included them in our chat. I'm not sure that this feature was that valuable in this case, but it seems that textually generating SVGs is now a product feature at least for ChatGPT. The SVGs had some clear issues and I found myself wishing that it had done a better job: the nose of the rocket is overlapping with the rocket body, and in the second image I think there's only supposed to be one orange line but it tried to add styling or something that went wrong.
- I was having trouble embedding the SVGs as links, and ChatGPT recommended Simon Willison's SVG Renderer as a workaround.
Just my 2 cents on the pelicans benchmark: I think it might help if you were a bit more explicit about not taking this one benchmark too seriously. I know you say that at times, but at other times (such as the comment I'm replying to) you elide it which can read as a seriousness about the benchmark.
cole_h06 2 days ago [-]
[flagged]
nicolamanzini 2 days ago [-]
[dead]
zzulanas 2 days ago [-]
what is even the point of these now? labs are most definitely training on this question now. I feel like this test scratches the surface of a "real" test for these new models
simonw 2 days ago [-]
It attracts at least one comment like yours on every single thread, so I guess the main purpose is to kick off a race to see who can post that comment first.
32297 2 days ago [-]
Yeah the main point of a 120 DB noisy party at 4 in the morning is to see which neighbor complains first and then make gleeful comments about it!
hyperparticles 2 days ago [-]
The roof is always raising month over month. If you were to make a harder version of the benchmark, what do you think you'd like to see the agents do better?
boo_you 2 days ago [-]
it actually makes a lot of sense that this is an elaborate troll account
anthonyrstevens 2 days ago [-]
People who are unable to comment on the underlying technology or capabilities because they don't know or don't care to inform themselves sure can feel good that they have an opinion on pelican art.
algoth1 2 days ago [-]
I enjoy it. Many readers here look specifically for Simon's pelican comment when a new model launches
129751 2 days ago [-]
Memes and self-promotion.
HelloMcFly 2 days ago [-]
It's merely become part of the culture of new model drops even as it's become a bit less useful as a diagnostic.
soperj 2 days ago [-]
low version doesn't have a chain.
Medium and high are fixies apparently, (or they have a worm gear, although the hub doesn't look big enough for that), since neither has a derailleur. That would also explain the lack of brakes at least.
Low is still obviously failing the "test". It's pedals are not actually attached to the bike either.
Medium has a handlebar that goes through the pelican's beak. So I'd say also failing.
For the high, the only nitpick would be that the bell isn't quite on the handlebars and the stripes on his scarf don't line up correctly.
staticman2 2 days ago [-]
It seems to be a deadpan inside joke around here that everyone pretends to care about the pelican test.
make3 2 days ago [-]
The models are solving Millennium Prize questions and making progress on the Riemann Hypothesis, yet the top comment is still looking at pelicans somehow. To me, it feels even more superficial and pointless than arguing about the presidential candidates' tie choices in a debate.
sashank_1509 2 days ago [-]
I’m curious at the enterprise level does Anthropic have staying power. My sense is when you use it with your own custom harness, it doesn’t, I can switch between Opus 5.5 or Gemini Argon and not really care as long as capabilities are similar. This is good for the consumer, and bad for the model developers.
seer 2 days ago [-]
Hmm how does your custom harness compare to Claude’s?
I’ve tried the various alternatives (opencode, pi.dev) but for “have several physical machines, each running an orchestrator and 5-10 agents, and each agent spawning its own subagents to do work in parallel, all of that on a tanky GitHub project boards” level of work only Claude was doing work dependably.
I can have a few orchestrator leads which I talk to and brainstorm, and let them file tickets, organize them and assemble teams of agents to work of the various problems. And all of that with just a smidge of custom tooling on top of Claude Code’s default.
Plus don’t have a budget to spin all of this on cloud runners so I made sure to do as much as I can with local work on several machines I own. Was quite easy to achieve with Claude.
Do you have something similar built on top of a open source harness, or is it from scratch?
sashank_1509 2 days ago [-]
Well I work at BigCo and I have no idea how it compares to Claude Code because I’m not allowed to use Anthropic models outside of our custom company harness. Opus 5.5 works extremely well, it can launch multiple subagents, assign them tasks, use our company’s inscrutable build system (that no one really understands anymore), and run tests, all by exposing harness commands in the system prompt. I don’t think we are lacking in any way.
seer 2 days ago [-]
The switch in mindset I experienced was to go from "Working on a feature" to "curating a board" to "overseeing angets curating boards".
So that I'm not just relying on one or a few agents to accomplish a task. I work on the whole product and just drive the direction of tickets and their "what done looks like" requirements, and let the machine organize things on its own, usually 10-20 independent agents (each spawning their own subagents). I think i might have capacity to handle more, just don't have the subscription tokens to fuel it and the budget for the hardware to run it.
Haven't worked in BigCo so no idea how that compares to their tooling. I'm sure it works much more efficiently and on a bigger scale, just being curious about how it all plays out.
jbellis 2 days ago [-]
I built http://github.com/brokkai/mjolnir/ to do exactly this cross-harness over ACP. As a bonus it can let Opus drive Luna as a subagent, which is just ridiculously cost effective. (5x clst multiplier means Haiku is much more expensive for realistic task sizes.)
lowbloodsugar 2 days ago [-]
Please tell me how you do that without Claude forgetting its context all the time. I feel like luke fucking skywalker talking to r2d2.
claude: Help me, Obi-Wan Kenobi. You're my only hope.
me: who is obi-wan kenobi
claude: oh you have to log in again.
me: <logs in>
me: play the whole message
claude: what message?
seer 2 days ago [-]
To be honest I’m not sure - the tooling around it has processes to start things, and gourds around tool usage to remind Claude of proper procedure.
But ultimately I find it a silver lining more than a problem. I am building and changing tooling so fast, and the models tooling itself is changing so fast, that by the time I establish a workflow, I either A) have solved the original problem so no longer needed, B) have built other tooling that distorts the picture, or C) the labs have released something new and cool that changes the equation drastically so your previous setup is gone anyway.
For example I built some procedure on how to split work between fable and opus and sonnet to optimise token usage, then opus 5.5 came along and made all of this obsolete.
Not per “token usage” mind you, opus is still expensive, but more “effort per task time finish” kind of metric. If I want to close things faster while still being relatively within budget, it is just easier to send all to opus - it will now just solve thigns quicker so you still use similar amount of tokens / money, but quicker, which is a trade I’ll make any day.
And every time “the system is working” I either just stumble onto the next resource constraint, or I feel uncomfortable and start refactoring to make things better and inevitably destabilise the whole “machine”
solenoid0937 2 days ago [-]
It depends on how expensive the model is to serve. If it turns out OpenAI and Anthropic can serve the models for a fraction of the price versus Google/Meta, then Argon/Muse remain a curiosity from the investor's perspective.
We won't know the answer to this until IPO and until each company reveals the economics of their models (training and serving).
avinoth 2 days ago [-]
One thing I've observed Gemini being the best is in the context of non-western stuff. Like, when I ask about obscure srilankan food, or chat about Indian/Tamil movies or songs, Religious customs & beliefs, Gemini almost always answers accurately and very close to how helpful they are in Anglosphere related topics. ChatGPT also does do a bit better recently, but Claude/Grok are absolute trash on those fronts.
Google's years of research project into Indic scripts, image trainings, translation efforts & transliteration efforts is literally paying off. The AI adoption from non-English, non-Chinese population will only if you have these contextual localisations, which I think Gemini is poised to do well.
For example, you can ask for a shopping list for a marriage function in a town from Tamil nadu (the marriage custom varies by districts), and Gemini provides contextual and helpful answer (like what is the custom followed there, what is gifted traditionally, etc), whereas if you ask Claude/ChatGPT, it talks about marriage registeries or pretty generalised answers which is not relevant or useful.
Hugada 8 hours ago [-]
That has always been my experience with Google and how they work their datasets. I think it's already well known how their picture post-processing is close to the best for people with darker skin tones as their post-processing models are trained with more varied datasets in an intentional way. Which is somewhat close to what you're mentioning here.
rootsu 2 days ago [-]
100% agreed. When I ask to generate an image or a poster with an indic language, chatgpt and claude always make gibberish words for something that is not at the center but gemini nails it almost every time in first attempt.
2 days ago [-]
sureglymop 2 days ago [-]
When you are on "gemini.google.com" there is a little lock icon next to Gemini 3 and Gemini 4. When you click it you get the prompt to subscribe (pay). Once you have successfully done so, the Gemini 4 option disappears!
Pretty misleading.
ezfe 2 days ago [-]
Maybe it’s just an incomplete rollout not fraud lol
verdverm 2 days ago [-]
don't attribute to malice that which can equally be due to incompetence
peheje 2 days ago [-]
Something can be misleading whether or not anyone meant it to.
verdverm 2 days ago [-]
I agree and do not think our comments are in contention.
a11r 2 days ago [-]
I seriously doubt their ability to double API prices on Dec 1. Token prices are in a clear downward trend and Google should maximize revenue by competing for volume. Currently OpenAI is the only US frontier lab trying to seriously compete with open-weight models that you can run yourself for an order of magnitude lower than what frontier labs want to charge
sidibe 2 days ago [-]
The goal is to move people along. By then they'll have something as performant but cheaper for them
mattmaroon 2 days ago [-]
I’m running 6.1 sol as my main driver on Hermes these days. It does so much for so little cost.
bitexploder 2 days ago [-]
It actually finishes things for you? It's been unusably slow for me. Worse than batch mode anything.
tireonfire 2 days ago [-]
I have the Free AI Pro plan from when I purchased my Pixel 11 Pro. I still don't have Argon in AGY nor web nor Gemini app. I have a free product and still feel cheated.
lp92 1 days ago [-]
Likely because of phased roll out. Seems like a few others on the Pro plan are reporting the same thing. I am on the Pro plan and I see it in the Gemini app both on PC and Android.
jazdw 2 days ago [-]
I'm on the same plan (from buying a Pixel) and it's available to me
JacobAsmuth 2 days ago [-]
Feel better now?
efficax 2 days ago [-]
Only available on Ultra, so I thought maybe I'll try it for one month to see if it's good. Well, I have Google One and for some reason this means I cannot sign up for AI Ultra? This is just one of the worst product rollouts I've ever seen
mchusma 2 days ago [-]
Website says its available on Pro too, but i have pro and can't access it.
ketzu 2 days ago [-]
I checked gemini, set it to "gemini 4 argon" and it offered me Pro and Ultra to get access. I checked the normal plans, saw there is also plus.
Okay, so I'll get Pro to try it.
I subscribe.
No gemini 4.
The most generous reading I have is that it's a mistake. But I also downloaded the PC app, and now I really wonder why I did that, it seems to offer nothing beyond the web experience and also doesn't even know its a local app. (When asked about its capabilities it always responded with reasons pertaining to it being a web app.)
lp92 1 days ago [-]
Its available on the PC Gemini app under the Pro plan for me. But not available in Antigravity.
ketzu 1 days ago [-]
After checking again, now it is available as well. Same as for you, it doesn't show in jules and antigravity.
However, my experience with the gemini, jules and antigravity apps was awful.
So it is difficult to pin that on the model.
I wanted it to give me a proper comparisson for VPSes or server options and it took so much babysitting and it couldn't produce reliable pricing information or visulalisations.
richrichardsson 1 days ago [-]
Pro subscriber here: available in the Gemini app, not available in AntiGravity.
I'm not going to 5.5x my subscription cost without being able to try it out first. To be fair though, I'm not going to 5.5x my subscription cost anyway, as my current usage would not warrant that! But that doesn't change the validity of my prior point.
llm_nerd 2 days ago [-]
It's available to Pro in the Gemini app (and the gemini.google.com). It isn't available in antigravity. They know that the people who are using these tools for coding are most likely to cough up for the Ultra subscription.
EDIT: Okay, the AI plans page claims under pro - "Expanded access to Gemini 4 Argon in Google Antigravity." So maybe it is just a matter of it rolling out. Though Google has a long history of serious confusing and misinformation on those tier descriptions.
EDIT2: And they changed it again, and now it is explicitly excluded from antigravity for Pro users.
129751 2 days ago [-]
Google's models are by far the best for readable text but also for nuanced understanding of the input.
Even after using the free model in Google search talking to Anthropic or OpenAI models feels like talking to an ox.
Christine725 2 days ago [-]
Gemini’s writing is really good, but unfortunately it hallucinates too much and doesn’t follow instructions very well. If only Gemini and Claude could be combined into one.
FranciscoCarlos 2 days ago [-]
Gemini 4 Argon
As of my knowledge cutoff, three of the top state-of-the-art large language models are Google's Gemini 2.5 Pro, OpenAI's o3 (and GPT-4.5), and Anthropic's Claude 3.7 Sonnet.
parasti 2 days ago [-]
Knowledge cutoff isn't really what these models are used for.
FranciscoCarlos 2 days ago [-]
How is a model that has never seen an agent on it's pretraining data going to reasonably plan and fan-out tasks to multiple agents? Or engineer software for agents to edit semi-automatically, it has zero awareness of it's capabilities in it's internal representation.
akoboldfrying 2 days ago [-]
I don't think that's likely to be a problem, really. If I explained the concept of an agent to you in a couple of sentences in Feb 2025, could you understand it, even though agents didn't exist yet? I think you could, and I think a modern frontier LLM probably can too.
FranciscoCarlos 1 days ago [-]
Yeah, it can be worked around with in-context instructions but they still tend to scope, delegate and architect without awareness of the fact that agents exist and work. It's an actual big deal IMHO, opus 5.5 is the most updated AFAIK.
piperswe 1 days ago [-]
Nowadays the scraped training data is only one part of the training process. The RL phase of training can instill a knowledge of tactics like sub-agent delegation without documentation about them being present in the training corpus. Context isn't the only way for a model with an early knowledge cutoff to know about it.
FranciscoCarlos 11 hours ago [-]
Still a crutch around incomplete pre-training data, a ton of useful knowledge wont be properly represented on the weights.
bender250 2 days ago [-]
Exactly my first test on all Gemini models. It cannot recollect any of the top events since Feb 2025. The worst about this is that since 3.6 Flash Google devs stated "knowledge cutoff March 2026" in the system prompt, so the model is wrongly confident that it knows about the year that it was not trained on. Which means that it searches less than it would.
And at least Flash 3.7/3.8 were both difficult to persuade that they have wrong knowledge cutoff.
I reported it as a bug to Google, but as all of my reporting to Google, no response ever. This is just stupid!
FranciscoCarlos 1 days ago [-]
They know it and they try to cover it up with a fake reported cutoff date. Claude reports June '26 but it has only partial 2026 data (tested it against the API and it's 25 as well). GPT refuses to answer but it's old 25 data.
arbuge 2 days ago [-]
You need to use search grounding.
FranciscoCarlos 2 days ago [-]
They need to pretrain on fresh data.
zobzu 2 days ago [-]
or, they've had 4 for a long time (which they did). AFAIK they have had 5 for a while now too.
I'm not quite sure what their strategy is, because it looks like the models released just have tweaks on the same original training data, but nothing more. Basically, they decide not to release them and it doesn't seem to be a much different when they finally to decide to release.
FranciscoCarlos 2 days ago [-]
Seems like a rushed RL run on top of an old model to jump into the hype train, seems reasonable since they hold a fantastic distribution moat. Even if it's not SOTA a billion people will use it and they'll collect data-points for future iterations. Same as muse.
I_am_tiberius 2 days ago [-]
I just created a Google account, which was a hassle. I tried to find an app and found AI Studio, but I’m not sure what that is. Then I found Antigravity, which apparently is Google’s alternative to Codex, and I installed it. Then I tried to buy the small Gemini AI Ultra package, but it didn’t let me pay. It accepted neither my credit card nor my PayPal account. It’s crazy how bad the Google experience is nowadays. You don’t know what to use or where to find it, and the security measures seem totally overkill. Am I really the only one having these experiences? Is Google aware of that?
dynm 2 days ago [-]
I bought a google play card (or whatever it's called) with cash due to these issues. Result: Instantly rejected + locked after entering the code, no credit, no dispute, no customer service, no refund.
rdtsc 2 days ago [-]
I am not touching it for that reason. One wrong step and the whole gmail account is locked without any recourse.
CamperBob2 2 days ago [-]
Time for small-claims court.
bluerooibos 2 days ago [-]
Yeah being presented with these options is such poor UX:-
- Try in Gemini
- Try in Google AI Studio
- Try in Google Antigravity
Will non users of Google products even know what these three things even are?
34679 2 days ago [-]
Also "available today for Google AI Pro and Ultra subscribers".
I have AI Pro, but AI Studio tells me I can "unlock" Argon by upgrading to Ultra.
ssernikk 2 days ago [-]
I have AI Ultra and agy wants me to upgrade to... AI Ultra.
MadameMinty 2 days ago [-]
"You've had one, yes. But what about second Ultra?"
rezahussain 2 days ago [-]
I dont think he knows about second Ultra, pippin
vessenes 2 days ago [-]
Don’t stress. I have ultra and the “try Gemini 4” only loads 3.6 flash.
IanCal 2 days ago [-]
Same - it doesn’t swap the model.
Classic Google.
malshe 2 days ago [-]
Same here. I had not opened antigravity in two months so I thought it was an update issue. But the app was updated. Argon is grayed out
irthomasthomas 2 days ago [-]
Same for me. Pro sub but the best I can access is Flash 3.6
DriverDaily 2 days ago [-]
Irrelevant. The target audience was Google VPs.
Lucasoato 2 days ago [-]
Same thing, I've always paid my bills, I have had some sort of google subscription for the last 10 years. Now they're requesting me to switch to pre-paid billing. They won't accept my card (that has worked every time this year so far).
Is there anybody in Google having a look at how bad and complicated their payment system is? In all the other AI providers you just sign in, pay, get a key, done. In other clouds like AWS, you add a card, pay little to verify the card the first time, monthly bill, done. In Google it looks like you have to fight them every time for the very thing they should make as easy as possible: opening your wallet.
gpugreg 2 days ago [-]
I'll believe in AGI when Google can figure out how to leverage their model to make their user experience not suck.
fsflover 2 days ago [-]
But will it bring more profit?
deno 2 days ago [-]
First install Antigravity and see if it works on Free with your account at all. You can buy Ultra and then find out you don't have "Antigravity eligibility" on a given account or it will ask you to send SMS to some nonexistent number.
I would also advise against buying Gemini if you have any other Google services, because you get banned across board if they don't like something. To use Google it's best to first migrate off of Google.
Otherwise great product, I really like and would recommend Gemini. The usage is also very generous, at least Flash models.
serbuvlad 2 days ago [-]
Side story/rant
I recently had to verify my YouTube account with a phone number recently because it requires this in order to set a thumbnail.
Questions right off the bat:
1. Why do I need to verify to post a thumbnail that will take 1MB on their servers but not to post a video that will take 100MB on their servers and be public??
2. Why do I need to verify my phone number on YouTube when it's linked to my Google account which has a verified phone number and 2FA etc.?
In any case, it takes you to a CSS-less page that asks you to choose between getting an SMS or being called to get a code. I choose SMS and it doesn't work. It shows an error message with no description just "something went wrong". I try a few times and it keeps not working.
I then switch to the call, and guess what! I have tried "too many times" and will have to try again later. Call worked the next day, but geez, what a mess.
Xenoamorphous 2 days ago [-]
Reminds me of that email from Bill Gates about trying to install Movie Maker.
codethief 2 days ago [-]
This is the first time I'm hearing about this email (thanks!) and I'm wondering: If it was already known in 2003 how bad Windows was, how did we end up with a Windows that's so much worse? Bill said it was his job to send such emails but apparently he didn't do his job very well…
Well, we knew about climate change at least in the 70s already, so...
jasondigitized 2 days ago [-]
It is mind boggling how difficult it is. I should be able to go to some well known URL, e.g. gemini.com and have the same experience I have on claude.ai or chatgpt.com. I can't wrap my head around why they are over complicating this. And it should largely feel this same as when I install 'Anti Gravity' which is just a crazy bad name. Why isn't it called Gemini Desktop?
dieortin 2 days ago [-]
> It is mind boggling how difficult it is. I should be able to go to some well known URL, e.g. gemini.com
What’s so complicated about going to gemini.google.com ?
> Why isn't it called Gemini Desktop
There is a Gemini desktop app. Antigravity is for agentic stuff, similar to Codex.
tyingq 2 days ago [-]
Of course, there's Antigravity IDE and Antigravity 2.0. Which are dissimilar.
guybedo 2 days ago [-]
yeah ...
i live in French Polynesia, a French territory, i'm a Google AI subscriber and i can't use Google AI Studio or antigravity cli because i'm in a geo restricted area ...
It seems Google has no issue laying out cables across the Pacific Ocean from/to Tahiti, they don't have an issue selling their services and plans there but i can't use AI Studio or antigravity cli because of geo restrictions ...
mfkp 2 days ago [-]
I have Ultra and went to try it and...
"Unavailable · Gemini 4 Argon is currently available to Ultra plan subscribers."
randerson 2 days ago [-]
I recently tried to create a new Gmail account for a small app I've made. I wanted a way for users to get hold of me other than my personal email address.
They now ask for your mobile number during signup. And it wouldn't let me register any more accounts with my number. So frustrating.
RGS1811 2 days ago [-]
- I clicked "try in gemini", typed a prompt, got an unimpressive result.
- Then I realized Gemini 3 was active.
- I tried switching models.
- I was presented with a $99/mo subscription offering.
- I scrolled down and clicked to get cheaper options.
- I chose the $19.99 "AI Pro" monthly option and paid.
- In chat I was able to use Argon. Cool. Seems pretty smart.
- I asked Argon if I could use it through opencode with my Google account.
- Argon said yes, but then told me I'd have to use the Gemini CLI as middleman.
- I told it I didn't know there was a CLI for Gemini; I'd just use that.
- It told me how to install gemini-cli.
- I authenticate via Google OAuth inside gemini-cli.
- Gemini-cli rejects my login, says the app is deprecated for subscription users, who must use agy.
- I install agy.
- In agy, I open the model selection menu.
- "Gemini 4 Argon is currently available to Ultra plan subscribers." Odd... Gemini 4 Argon is the model (included in my Pro plan) that told me to do this.
- I discover that Claude Opus 5.5 is included in the subscription. I switch to it and give it a debugging task.
- Within about 5 minutes: "Individual quota reached. Please upgrade your subscription to increase your limits."
- This is dumb. Google stole twenty bucks from me; I want my money back.
- I go to the Gemini web app to cancel my subscription. I paid on my phone, so I have to cancel on my phone.
- The Gemini iOS app has a "manage my subscription" page. I go there to manage my subscription.
- This "manage my subscription" page includes no way of cancelling.
- After a minute of confusion I go to the App Store > Subscriptions > Gemini > Cancel.
- My subscription will end at the end of the month (no refund).
- Then I have to find the complaint form so I can request a refund.
- But I did request one.
Anyway, "frontier intelligence" just wasted about an hour of my life.
JanTurnherr 2 days ago [-]
o brother, i had the same problem when i first wanted to use Gemini-Cli. after hours spent, a friend of mine and i did a direct comparison, he with claude-cli and i with gemini-cli. task was to create a small arduino cpp app. gemini was not able to produce even compilable code... waste of time and energy.
for me, a google model can be as good as they want, in the end, they are still google, probably one of the most bureaucracy riddled big companys out there. evrything i see from then gives me feeling that evrything has been decided by a committee that do care more about politics and not customer needs. the shit one has to go trough to get the gemini-cli running is a good example of that.
talon8635 2 days ago [-]
I agree 100%, it’s a mess
That said, asking Gemini.google.com to walk you through the steps seems to suffice. In fact, one of AIs big perks for me is walking my through kafkaesque systems like Google’s to cut the shit and get to the result
notatoad 2 days ago [-]
i remain convinced that google doesn't actually want anybody to use these models. they want to announce things, for the sake of their share price and their promotions.
but they don't want to share their compute with us. giving those resources to their ad targeting algos is more profitable than renting them to us. so they make it as hard as possible. they tried as hard as they could not to release a new pro model at all, but now that fight has been lost they at least need to make sure we don't use it.
kccqzy 2 days ago [-]
I think it’s a typical strategy for scammers to create a new account and then immediately try to buy an optional service with a stolen credit card.
What you did isn’t wrong, but is unfortunately very similar to what real scammers do. The heuristics behind these anti-abuse efforts are always going to have false positives.
the_real_cher 2 days ago [-]
Using google services has always been miserable. It used to be way worse. Now you can at least get an api key from AI studio without going through 20 esoteric steps to configure everything.
Lucasoato 2 days ago [-]
At least now they don't play the "client side key can now access your very expensive resources" game anymore.
framapotari 1 days ago [-]
At this point I just assume that it's impossible for me to try the new model that Google announces and invites me to try.
rumblefrog 2 days ago [-]
In the same boat, I have a single seat Google workspace account, and can't figure out how to add Argon to my workspace. Asking Gemini seems to make it hallucinate pages that aren't there.
radicality 2 days ago [-]
Also single seat workspace user. Half the time my Gemini forgets I connected Gmail to it and hallucinates false instructions to enable it.
Truly crazy how much Gemini still hallucinates, especially things which its own Google documentation or just a basic web search away.
I feel it’s mostly solved problem with chatgpt/claude , but for me, with Gemini it’s almost guaranteed it will just make stuff up
singingtoday 2 days ago [-]
Same issue here, didn't accept my credit card. Didn't say why either.
throwaway919233 2 days ago [-]
You didn't have a google account? Impressive feat
mrshadowgoose 2 days ago [-]
Nope. This is exactly the same way they fucked up when they had the model quality lead with Gemini 2.5 pro.
They made using it so hard/unclear that myself and every other dev I knew gave up.
It's clear that they still haven't done any organizational self-reflection.
_s_a_m_ 2 days ago [-]
I hate everything with Google and consumer facing services, they dont give a shit because 99.9% are monetized through spyware or are buisness customers.
They had gemini cli, what happened to that? Was fine
InsideOutSanta 2 days ago [-]
It fell victim to Google's long-standing strategy of building everything twice and then killing the better option.
AspireOne 2 days ago [-]
To be honest, Gemini CLI was disastrous. The Antigravity CLI is much better.
kristofferR 2 days ago [-]
Yeah, a month ago I wanted to test out Gemini-3.5-Transcribe, but had to wait a completely indeterminate time, got the offensive message "Your Google Cloud billing account is being processed. Processing time varies from a few moments to a few weeks.", like I'm just worthless scum. Took over a week for no reason.
Ended up with Soniox instead, took minutes to get started.
algoth1 2 days ago [-]
Just wait until you try to use the Google Cloud Console
0xbadcafebee 2 days ago [-]
If you're not trying to buy ads, you're not really a customer; you're a product they will sell to people trying to buy ads.
pietz 2 days ago [-]
I'm open to Google winning me over again, but there isn't enough hype in the world to do it. They will need to earn it after these disappointing 6 months.
swaptr 2 days ago [-]
I appreciate how it's being integrated into the Google ecosystem, but the user experience still feels unpolished when actually using the system. The marketing, however, is impressive.
chaostheory 2 days ago [-]
You probably need to wait more. Argon just keeps telling me that it “encountered an error” and to “try again”
tezza 2 days ago [-]
Put it through my standard benchmark, a six step explainer... yes, in 3D.
Very fast.
Makes mistakes. Corrects the mistakes quickly with direction.
Astra 6, you mean? 5.6 didn't have an Astra release, right?
karahime 2 days ago [-]
I would love to try it out, and the Gemini models had previously been some of the ones I'd liked best in an overall sense, but unfortunately their updated inference endpoint is no longer ZDR on OpenRouter (change takes effect on the 18th). I think this is one of the more unfortunate turns in AI development in 2026, as it seemed like labs offering it was becoming the norm last year, and Google had previously done so consistently.
anonova 2 days ago [-]
AI Studio has never been ZDR (retention is noted at 55 days max), but paid usage is excluded from training. Vertex uses enterprise terms, and it's definitely ZDR/no training. What's happening on the 18th?
karahime 2 days ago [-]
It's the Vertex endpoint for Gemini models that's changing to be reclassified by OR as not ZDR, as I understand it. The email from them says
> Upcoming change to OpenRouter’s ZDR designation for Google authored models
> We’re writing to let you know about an upcoming change to how OpenRouter designates Google authored models for Zero Data Retention (ZDR) routing.
> Effective October 18, 2026, OpenRouter will no longer designate Google authored models as ZDR-eligible. After that date, requests that require ZDR will not route to Google authored models.
> Google’s standard automated abuse-prevention policies allow it to use safety classifiers to identify requests that may violate its policies and to securely retain prompts flagged for further review. Because this process does not meet OpenRouter’s requirements for a ZDR designation, we are updating our model classifications and routing controls accordingly.
> What this means for you
If your requests require ZDR
We recommend migrating affected traffic to another ZDR-eligible model before October 18, 2026 to avoid interruption.
> If you do not require ZDR
No action is required for your Google model requests.
> Our current routing behavior will remain unchanged during the 30-day transition period ending October 18, 2026.
> Read the ZDR documentation
> If you have questions or would like help identifying an alternative model, please contact us at support@openrouter.ai.
> Thanks for using OpenRouter!
> The OpenRouter Team
staticman2 2 days ago [-]
If I recall correctly this is because OpenRouter is changing their definition of ZDR so that Gemini was never ZDR.
fusslo 2 days ago [-]
As I approach a transitional period in my career, I'm reflecting on the managerial effort of places like google. How on earth do they keep an engineering org on the 'right track'? I work on products that are not as complex: fewer moving parts, fewer egos, no where near as novel. It feels like nobody does anything and nobody has any expertise
is it just 'Hire the right people'?
serioussecurity 2 days ago [-]
It's a quarter million people (including temps/vendors/contractors) over almost 28 years. There's no "just" here.
It's extremely inconsistent internally as well.
Look up the slime mold analogy for Google.
boelboel 2 days ago [-]
Makes me wonder if more growth/innovation could've happened (outside of google) if they didn't employ this many people. Some sort of dutch disease within the higher echelons of the tech sector.
Eridrus 2 days ago [-]
Yes and no.
Google did a very good job on things that required longer research times and compute scale, note that Transformers, BERT & Scaling Laws came out of Google (though the latter were also worked on at OpenAI).
Conversely, Google is horrible at product, and without OpenAI, it probably would have taken many years longer to take off. I know this because Google had demos of this internally, but absolutely no stomach to release it, and they didn't believe in the API use case at all.
Which is to say, I think if Google was smaller, we probably would have gotten more SaaS, not more breakthrough research.
no_carrier 2 days ago [-]
Pre this whole LLM era, Googled would've been roasted if they created a chatbot that hallucinated to users. I feel like this was the main reason they never turned it into a product and then ended up on the back foot.
Eridrus 1 days ago [-]
Absolutely, Google lived in fear of that and so we basically needed someone with nothing to lose to break us out of that fear of the media.
Eridrus 2 days ago [-]
They haven't solved management and definitely don't keep things "on track" globally. However, they definitely don't have the problem that nobody does anything and nobody has any expertise.
It's also worth noting, Gemini 3.5 Pro never came out.
Having worked there (but only in a small part of it that eventually got absorbed into Gemini), it's some combination of:
* The eye of Sauron (management) can make specific spots (e.g. Gemini) push harder (and attract the motivated people at the company and be selective)
* They hire a lot better than average (Generally from the top 10%, hotter places in Google are more selective)
* They have a promo process that gives career motivated people a way to prove themselves (not always aligned with things being on the right track ofc, and not always non-political, but they definitely do something)
* Redundancy in teams causes them to compete (not necessarily productively)
Of course, there are other downsides not listed here (insane bureaucracy for launches, GDPR/BCID compliance, etc, etc), but a lot of what makes Google possible is the insane amounts of cash Search throws off.
arjie 2 days ago [-]
It’s like the USA. How do they keep people on track? They don’t. You delegate and hope things work. When they have Google Ads and Search they can experiment a lot and “waste” a lot in search.
estearum 2 days ago [-]
Bottomless buckets of money produced by their web monopoly so they can continue on at pace, virtually regardless of how inefficiently they do it.
2 days ago [-]
jacquesm 2 days ago [-]
I think the google graveyard is a partial answer to that question.
IncreasePosts 2 days ago [-]
Google has tried a lot of ideas and then abandoned them, but has it fumbled any already successful product?
Perhaps one of their more successful chat clients, but that's all I can think of
jacquesm 2 days ago [-]
Success for google is defined in a different way than it would be for an ordinary company and many of those would have been successful if they had started outside of google.
The really annoying ones are the ones they buy, and then shutter because they can't make enough money on them fast enough.
braj0 1 days ago [-]
RIP Google Nest CO2 and Smoke Detectors T.T
I thought the nightlight that goes on when you walked under it was really neat...
jacquesm 1 days ago [-]
The real problem is that they put a lot of parties making similar devices out of business because they could not compete with google on volume, pricing (subsidized, of course) and marketing.
ceejayoz 2 days ago [-]
> How on earth do they keep an engineering org on the 'right track'?
What if the answer is "they don't"?
ur-whale 1 days ago [-]
> How on earth do they keep an engineering org on the 'right track'?
They don't.
If they did, neither OpenAI nor Anthropic would even exist, as the core of what they're all doing was invented by Google engineers and researchers.
asdewqqwer 1 days ago [-]
One thing to notice: Now, even in EU, if you turn on Google activity (so that you can see chat history), Google will train on your data. I remember it wasn't the case when in 2.5 pro period.
damsta 2 days ago [-]
> The model launches today at an introductory price of $2 per million input tokens and $10 per million output tokens (with cached input tokens at 95% off).
Couldn't you just say what the price will be after the introductory period and when it'll take effect? Do they have some SKILL.md for these blog posts that requires the most important information to always be buried ten levels deep and hidden behind a login page?
$2/million input and $10/million output today. $4/$20 after December 1st.
nateb2022 2 days ago [-]
With the pace they're delivering models, there's gonna be 1-2 new Gemini 4 versions by December 1st, presumably with their own introductory price windows. We just got Gemini 3.8 on 2 September and Gemini 3.7 on 13 August.
gcanyon 2 days ago [-]
This reminds me of telephone commercials that ran back around 1987, saying they were offering long distance calls for 10 cents a minute (a pretty good price back then) and -- this was emphasized, a huge selling point -- they guaranteed your rate would never go up. In retrospect, all the lols :-)
vineyardmike 2 days ago [-]
I wonder if that’s more to price in the cost of keeping old models around on expensive inference hardware. It’s not like old models get cheaper to run, and a more heterogeneous fleet is less efficient to manage.
p_l 2 days ago [-]
[dead]
squidbeak 2 days ago [-]
Their last non-flash frontier release was 3.1 Pro back in February. Two new frontier releases at that pace would be Feb 2028.
mlmonkey 2 days ago [-]
December 1st is just ~6 weeks away. There's no way Google puts out even 1 more Gemini 4 version before then, let alone more than 1.
vikramkr 2 days ago [-]
No they've been releasing models super rapid fire. Other companies too. 6.1 sol was barely a week after 6.0 sol
mufufu 2 days ago [-]
Don’t underestimate the power of exponential growth
heaney-555 2 days ago [-]
You're conflating Gemini Flash with Gemini Pro.
damsta 2 days ago [-]
Opus 5.5 is also priced at $4/$20, but are they on the same level?
fancyfredbot 2 days ago [-]
At that price I think you'd pick Opus 5.5.
It's perhaps a face saving thing to pretend you'll charge the same. Let's see if it actually happens.
logicchains 2 days ago [-]
>$4/$20
Is Elon Musk doing their pricing?
square_usual 2 days ago [-]
That is the standard pricing for Opus-class models these days.
eptcyka 2 days ago [-]
Blaze it, Robot!
LeBit 2 days ago [-]
4/20. It is to Musk what 67 is to children.
harambae 2 days ago [-]
I think it's more 420 or 4:20 (marijuana reference) than 4/20 (Hitler's birthday).
Although given it's Musk, others may disagree.
WhereIsTheTruth 2 days ago [-]
There is no such thing as "standard pricing" for products with different performance profiles, architectures, inference costs and infrastructure stack
It is price fixing, nothing else
ac29 2 days ago [-]
GLM 5.3 is roughly Opus class and is much, much cheaper
wasfgwp 2 days ago [-]
It’s more expensive than 6.1-sol though? At least according to benchmarks.
I think at this point comparing raw token pricing is often apples to oranges.
denismi 2 days ago [-]
Those "price per task" benchmarks seem to pre-suppose the existence well-defined requirements and acceptance criteria.
When that isn't available, and the task involves "understanding the problem", "deciding what I'm actually after" etc. iterations, I find price-per-token to still be a better predictor of cost.
vikramkr 2 days ago [-]
Opus 5 class yes, but opus 5.5 is functionally mythos tier
simonw 2 days ago [-]
Opus 5.5 class?
ac29 2 days ago [-]
No, since that's a moving target. But the large GLM models are likely in a similar model weight class to the Opus family, just a few months behind
girvo 2 days ago [-]
No, and anyone who says otherwise is being silly.
It’s still damned capable though, I’ve done a crazy amount of good work with 5.3 Flash.
mrandish 2 days ago [-]
> Couldn't you just say what the price will be after the introductory period
+1. FWIW, I didn't even get far enough to wonder about the price.
The lead (and only) sentence states "Our next era of frontier intelligence, built to solve your hardest problems" a statement equally true of any new Google or competitor model, hence completely devoid of useful meaning. I then kept scrolling down... and down... and down, giving up after a half dozen screens of big images, huge videos and some customer stories. The first sentence of every text block was equally vague or meaningless so... tv;dr (Too vague, didn't read).
I mean its also been generated by a bunch of PM that want to get promoted but have little idea of what they're talking about..however nobody has been fired YET for using too much vague AI words.
nananana9 2 days ago [-]
It's irrelevant information, they know the reader switches models every week.
dyauspitr 2 days ago [-]
They don’t really. Most normies which are the vast majority of people just use ChatGPT. All tech folk just use Claude.
brainwad 2 days ago [-]
Normies aren't the target market for Argon anyway. The Gemini app is still serving 3.6 Flash and 3.1 Pro.
dgacmu 2 days ago [-]
As of now Argon is available in the android app, at least.
stymaar 2 days ago [-]
> They don’t really. Most normies which are the vast majority of people just use ChatGPT. All tech folk just use Claude.
It was the case a year ago, but it's less clear cut nowadays, I've seen non-tech people using claude on their phone in the train, and many tech people seem to have switched from Claude Code + Opus to Codex + Sol over the past few month.
staticman2 2 days ago [-]
Sometimes it seems like ChatGPT and Claude are releasing models every week.
dyauspitr 2 days ago [-]
Yeah, then I guess the GP was technically right it’s just people aren’t jumping providers constantly is what I was trying to get across.
vikramkr 2 days ago [-]
Normies are not the target for a frontier model
andriy_koval 2 days ago [-]
they are target with 6 month lag. Say, if current Gemini will be demonstrating obviously superior perf compared to Opus, many normies devs will switch within 6 months.
vikramkr 2 days ago [-]
Deva are not normie users by any stretch of the imagination, and they won't switch to a 6 month old frontier model because those will still be as expensive - they'll be using whatever the sub frontier model is at that time
apatheticonion 2 days ago [-]
Me, watching the AI world from the sidelines while using DeepSeek at $0.15($0.003 cached)/$0.60 - still bitter about the pricing changes.
It might not be as smart, though my employer gives me access to Claude, so I tried giving the same task to Opus 5.5. Both DS4.1 Flash and Opus5.5 produced nearly identical outputs.
After that, my company switched the entire engineering department to Azure hosted DeepSeek4.1 and costs were cut by 3/4. So far, no one has really noticed the change.
eyegor 2 days ago [-]
I'm surprised no one's complaining about it randomly switching to mandarin. It seems like no matter what I do once every few days it'll go back to mandarin mid thinking trace and then stays in mandarin until I remind it to use English.
apatheticonion 2 days ago [-]
It's just a subtle reminder to learn mandarin, haha
jklinger410 2 days ago [-]
Obfuscating token cost is the new great marketing frontier
mattmaroon 2 days ago [-]
They do, just not in that sentence. You must really hate sales ads.
Seems to be just deleting anything that a safety filter triggers on. I don't know if I like this more or less than what Anthropic does where they train the model to actively refuse.
Christine725 24 hours ago [-]
Gemini’s system has always been like this. I used it heavily from last September through March this year, and have been using it on and off since then — this has been the case since at least Gemini Pro 3.1. And that’s not even the worst of it. When its system detects something it deems “seemingly inappropriate,” it just freaks out. What I’ve personally experienced includes: (on the web version) wiping Gemini’s previous conversation content — once; and outright disconnecting Gemini from all prior conversation history, meaning a complete memory wipe — eight times. All without any notification. My conversations were pure text, just everyday chat about daily life and how I was feeling, absolutely nothing that would count as a violation in any general sense. But the system would just randomly lose it. After eight consecutive memory wipes, I mostly stopped using Gemini. Hopefully it becomes better now.
martythemaniak 2 days ago [-]
Pretty insane that this isn't available for Pro subscribers on Antigravity, like not even a taste. Antigravity comes with a sample of Opus 5.5 too!
lubujackson 2 days ago [-]
Available to me in Cursor. Google is too big to serve their own clients first, I guess.
elohimus 2 days ago [-]
I'm a bit overwhelmed by Google's selection of AI environments (Antigravity, AI Studio, Gemini Platform etc.). If I just want the equivalent of OpenAI Codex, which one do I use?
orloffm 2 days ago [-]
That's exactly Antigravity.
DuncanCoffee 1 days ago [-]
But is it the Antigravity app, cli, plugin, IDE or SDK?
You'll need Ultra subscription to use Argon in Antigravity.
eyalitki 2 days ago [-]
I gave the model my usual question of "I got a raise from X to Y, please explain what is the impact on my net salary". Answer was detailed and nailed all payroll edge cases, only to miserably fail in basic math at the end. It tried to calculate 430 minus 60 and got 410.
arjie 2 days ago [-]
Announced two weeks ago and they went on a victory lap and they're already behind by the time they launched. Strange company, Google.
Excellent model. Same price and score as Opus 5.5 in our 23 question exam.
This size / class of model is now a bit overpowered for these short-horizon data analytics tasks. OpenAI 6.1-Sol, Haiku 5.5, GLM 5.3 Flash all achieve about the same results but for ~1/10th the cost.
In other words, we're getting to the stage where models are "good enough" for certain tasks like data analytics and competitiveness is going towards size & speed.
2 days ago [-]
blurbleblurble 8 hours ago [-]
I tried it out and ran out of weekly usage on a $200 plan within a day. Not impressed.
dom96 2 days ago [-]
Adversarial programming language test for Argon. Surprisingly its safety guardrail gets triggered a lot just like for Fable 5.1.
Claude Opus 5.5 58.7%
Gemini 4 Argon 47.1%
Claude Fable 5.1 46.7%
>"Argon helped the Chrome team separate core webpage handling from a ~6 million line legacy folder of browser features held within Chromium’s 40 million-line codebase. Piping multiple clean code changes at a time, Argon helped the team finish an architectural refactor across hundreds of files, completing months of work in weeks."
Now that needed to be done for some time!
Regardless of if an AI helped or not, regardless of whether that was Google's or some other AI or not (though Argon does sound pretty cool!), regardless of whether an AI cut the time from months to weeks or not, refactoring Chrome -- decomposing it into individual clearly demarcated non-tightly coupled software parts with core webpage handling at the center -- that needed to be done for some time!
Now all of that being said, Gemini 4 Argon does sound pretty cool, and we congratulate both teams at Google -- the team that created Gemini 4 Argon, and the team that refactored Chrome with it!
But all of that being said, while the advancements made for Gemini 4 Argon are important and impressive (and they are!)... refactoring Chrome properly (IMHO) is slightly more important and impressive!
(You know, one man's humble opinion! That is, I am not the guy who is responsible for awarding performance-based bonuses at Google at the end of the year, so take everything I say with the proverbial "grain of salt"! :-) I am probably biased, one way or the other! :-) )
Insanity 2 days ago [-]
> Across Google and our early access partners, Argon is quickly becoming the preferred model for engineers tackling long-horizon software engineering.
"Across Google", wonder how much that is because of an effort by Google to curb usage of Anthropic models. MSFT and Meta similarly are pushing for in-house models. It doesn't mean the models are better when you push your engineers to dogfood and restrict access to other LLMs.
(Not that I know for sure this is the case, it's more of a hypothesis).
summerlight 2 days ago [-]
It's actually the inverse, they made Opus 5.5 available internally for all engineers, presumably for benchmarking against Gemini 4.
lp92 1 days ago [-]
Companies dogfood because that's the only way to get good feedback internally. What you or I say on the internet doesn't matter to these companies.
nkg 2 days ago [-]
Explain to me what is "long-horizon software engineering" in 2026.
melasadra 2 days ago [-]
I mean, don't we want engineers to dogfood their own product so that they know it's strengths and weaknesses and be in position to improve it?
bezko 2 days ago [-]
can't use in agy with Pro subscription:
Unavailable · Gemini 4 Argon is currently available to Ultra plan subscribers.
staticman2 2 days ago [-]
They just removed 3.1 Pro from antigravity pro so I certainly hope Argon will be added soon
yoz-y 2 days ago [-]
Excited to finally see this out. When new big models come out I like to make them audit any other code I generated to see how things are.
SilverSlash 2 days ago [-]
I have been very critical of Gemini in the past but one thing that I've felt Gemini has always been best at is being the most empathetic model imo. It feels far less cold and emotionless than GPT and Claude.
MaxikCZ 2 days ago [-]
I want my ai to be as cold as possible. I dont want a friend, I want a tool. I dont want my tool to pretend it has emotions.
Like when Miller in The Expanse talk to phone AI (https://m.youtube.com/watch?v=SC3bpJmGnPo) to trace Anubis flight, notice how the phone doesnt even reply in words.
Imagine if the phone instead replied with: "Thats a great insight, Anubis really goes there", it would be a joke. We are only tolerating it now because everyone is doing it.
lp92 1 days ago [-]
Excellent timing. I've been working on a UAV design and simulation tool and primarily using Flash 3.8 for the code gen grunt work because Claude hits weekly limits way too quickly. It's been working well this way. Flash 3.8 seems very efficient and i don't hit the limits before the refresh. Assuming Argon is similarly efficient then hopefully it means I can stop paying for Claud and switch over fully.
Trusteando 2 days ago [-]
Right now there are two top posts one about GLM-Legal and one about Argon. Both score about the same in the Harvey Lab test, but the prices are $0.15 versus $10.55, the $0.15 is Vals AI's estimate using the base model since GLM-Legal has not published prices.
dyauspitr 2 days ago [-]
Yeah, but one is a general purpose model, and the other one is specifically geared towards the legal field. Considering Opus 5.5 only gets a 4% on the test compared to Gemini’s 20%, that’s actually pretty incredible.
algoth1 2 days ago [-]
Gemini 3.1 is still SOTA at weird handwriting recognition. If Argon is better it will be amazing
spwa4 2 days ago [-]
I bet that's one of those areas where a finetune of a 4B model will wipe the floor with anything else.
matt123456789 2 days ago [-]
For readable or normal handwriting yes, big models are overkill. But weird handwriting is mostly edge cases. Bitter lesson territory.
spwa4 2 days ago [-]
If you can detect -> finetune -> rollout it doesn't have to be a problem.
simonw 2 days ago [-]
Pricing is interesting. At launch Argon is the same price as GPT-6 Sol, half the price of Claude Opus 5.5 and 1/5th the price of Claude Fable 5.1 and GPT-6 Astra.
But on December 1st Argon doubles in price - so it will be the same price as Opus 5.5.
mdasen 2 days ago [-]
I've kinda stopped looking at token pricing. Argon might be the same token pricing as Sol, but in Artificial Analysis' testing, Argon costs 5.8x more to run their coding tasks and 2.2x more for their general index.
Likewise, you look at Artificial Analysis' general index and it cost $8,708 to run with Opus and $2,407 to run with Argon so even if Argon's token price doubles, it won't be the same price as Opus.
Sonnet is cheaper than Opus per-token, but in their coding tests Sonnet burns through enough tokens that it's 7.5% more expensive.
So much depends on how token efficient something is and whether it gets you a good result or not, right?
Plus, if one is using a plan instead of paying token rates, what does my $200 get me from OpenAI, Anthropic, or Google?
It just feels very hard to compare pricing given that the number of tokens used can vary so much. In Artificial Analysis' coding tests, Sonnet used 7.1x more tokens than Sol. Sonnet and Sol might have the same per-token pricing, but there's a huge difference in the number of tokens used.
simonw 2 days ago [-]
Yeah, this is one of the most interesting challenges around model price comparisons.
This was hardly a consideration a year ago, when many model tasks could be tested using a single prompt and it was hard to spend more than a dollar.
meerita 2 days ago [-]
All AI pages now include canvas animations. There isn't a single page without one. This is neither good nor bad, just an observation. I think OpenAI has the best animations so far, with Anthropic in second place. This one looks a bit off.
ckx 2 days ago [-]
Best comment on this whole post! Thats a really interesting observation. Personally I think they look cool but I can understand how some people dislike them.
parallax_error 2 days ago [-]
Maybe they could ask Gemini to fix their awful website to not crash on my phone!
throwaway219450 2 days ago [-]
This website is absolutely terrible in iOS Safari, super glitchy.
I'm a bit confused, all the announcements say that this is availabe to Ultra and Pro subscribers, but in antigravity, it's grayed out and says it's only available to Ultra.
RedCardRef 2 days ago [-]
Available on Ultra and Pro on the Gemini App. Only Ultra in Antigravity.
From what I saw online.
rcr-anti 2 days ago [-]
Digging in the model card, "Gemini 4 is the next generation in the Gemini series, a suite of highly-capable, natively multimodal, reasoning models. Gemini 4 Argon delivers frontier performance..." If they're using noble gases there are 2 obvious lighter naming candidates and 3 larger (or 4 if you count Oganesson). I'm not seeing them claiming Argon is the top/biggest/bestest model among the Gemini 4 series, just that it has frontier performance.
Edit: noble gases not noble taxes, though the latter would be a funnier naming scheme.
Razengan 2 days ago [-]
[dead]
2 days ago [-]
bionhoward 2 days ago [-]
I feel like if Google simplified their privacy/model training policy then their stock would go up.
It feels so complicated and hard to know what we can use vs what hands them our codebases
axblount 2 days ago [-]
Gemini isn't perfect but I find it a lot less annoying than other model families (personal preference or stockholm syndrome). Interested to read the reviews.
Is it actually available?
robotmay 2 days ago [-]
Seconded. Anthropic feel the need to limit how abusive you can be to Claude because god, I want to throttle that snotty little shit. Google don't have to do that with Gemini because it just isn't as irritating. It does what I ask and doesn't try to second-guess my requests, or do "the quick fix".
chpatrick 2 days ago [-]
I'm the opposite, I found Gemini so annoying that I had to stop using it, Claude was a breath of fresh air afterwards.
hiq 2 days ago [-]
Maybe it's all about how often we encounter the quirks of specific model? Or simply that one's way to prompt triggers more annoyances with a model than another?
I'd be curious to see if people have managed to just add custom instructions to make the outputs more standardized and less annoying. I never have such annoying cases, whatever the model, it's just too verbose, but in this case I ask it to be more concise and it's usually fine.
chpatrick 2 days ago [-]
I wouldn't even call it a quirk or a chance encounter, every response it gives is sycophantic and overwritten.
IshKebab 2 days ago [-]
To be fair they dramatically improved how annoying Claude is with Opus 5.5. It's not perfect but it's not nearly as awful as it was.
artdigital 2 days ago [-]
Opposite for me. Output of 5.5 is better but its safety and moral standing skyrocketed. I am constantly hitting refusals for simple tasks, which previous generations (like 4.8) do without giving me a tirade.
Anthropic’s vision of “safe” is so frustrating to deal with, it makes me want to go back to ChatGPT.
lp92 1 days ago [-]
Same experience with Opus. It threw a fit when I asked it about some specific UAV questions for a UAV aero sim I'm working on.
dizhn 2 days ago [-]
I like the way it speaks and its "personality" too.
ckx 2 days ago [-]
I've been using ChatGPT since GPT 3 and even after trying different AI models, I always end up drifting back to ChatGPT.
alex1138 2 days ago [-]
It's been really bad at lyrics but gotten better. You look something up and it will just make something up and it takes a bit of digging to discover it's because of copyright
copperx 2 days ago [-]
"Gotten better"? Didn't it just get released a few hours ago?
alex1138 2 days ago [-]
I mean, I was talking about the 3 series
copperx 2 days ago [-]
Why would you comment about a completely different model?
kridsdale1 2 days ago [-]
Branding is a powerful force.
demibabs 2 days ago [-]
The parent commenter was also talking about the Gemini series in general, come on.
protoman3000 2 days ago [-]
Of all models I like to write with Gemini the most. The language appears much more natural and elegant than what even Opus 5.5 or Astra do. Also, they give 5TB of cloud storage and discounts, it’s a steal for what you get.
The code is bad though.
SwellJoe 2 days ago [-]
Claude has particularly bad prose. Somewhat improved in 5.5, but it has historically clearly been the most annoying prose among major models because is had so many obvious tics that made it clear it was Claude rambling. Still extremely verbose.
They all ramble, they're all allergic to getting to the point. But, Gemini historically has pretty good prose. Even the self-hosted Gemma models have pretty nice, inoffensive prose. I wouldn't want to read a novel written by any of them, but I tend to ask Gemini questions in chat more often than Claude or GPT, because it has good prose and is good at using the web to research.
mavamaarten 2 days ago [-]
Agreed. It's much better at working with stuff (like documents and presentations) in other languages too. It feels like it's trained in multiple languages from the ground up rather than as an afterthought.
jug 2 days ago [-]
Yeah rumor ahead of release was also that some Googlers were anxious about the known subpar coding given modern trends.
bahmboo 2 days ago [-]
Interesting tidbit but TFA and discussion are about Gemini 4
copperx 2 days ago [-]
Argon code is bad? Did you already come to that conclusion?
yipinwong 2 days ago [-]
Gemini models lie too much. Prentending something wrong to be right. You gotta treat Gemini models with disrepect, and even threaten to make it work.
Or else it say, here is the result. (but no it didn't do no research, and tells me, sorry I forgot to search).
It's got balls to lie straight at my face.
2 days ago [-]
TomGarden 2 days ago [-]
Same, gemini for writing is the strongest. Qwen was also surprisingly good last time I did a shootout
anon373839 2 days ago [-]
Qwen Flash Next has great writing! Unless it comes into any contact whatsoever with Claudeslop - then it immediately contracts the disease. It’s kind of irritating when you have it look at library code that’s been touched by Claude, because then the rest of your session is infected with it.
gradus_ad 2 days ago [-]
Agreed from my experience 3.8 Flash writes better prose than Opus 5.5
iamheretowin 2 days ago [-]
argon seems to be really good at spatial reasoning, it's catching really minute details which other models like astra and opus seem to miss.
I do wish I could give it a try with some free credits but I would bet Google is wanting to limit usage until they know they can handle the traffic.
I'm starting to feel the subscription fatigue and a small amount of FOMO. I just can't justify pro accounts for every frontier model now that it includes OpenAI, Anthropic, Meta and Google.
nitinreddy88 2 days ago [-]
not available yet. why to launch something that can never be used even after weeks
modeless 2 days ago [-]
Page says it is now available to Ultra and Pro subscribers. Although in AI Studio it currently says you need Ultra. But it's technically released either way.
bigyabai 2 days ago [-]
Because Mythos started an arms-race for "models that are too smart for you" and other labs are jealous.
theplumber 2 days ago [-]
Yeah, I think this is it! Google tried to hype their model but they have no chance against queen Dario. Perhaps they should just delete Gmail and say that Gemini did it. That way they would get some attention …
bigyabai 2 days ago [-]
Quoting Kai Lentit: "Google agents have been hacking Google internally for years."
It shows a good improvement on my COCO object detection benchmark: 0.504 mAP zero-shot.
Gemini 3 Flash was around 0.403, 2.5 Pro was around 0.34.
ralusek 2 days ago [-]
3.1 Gemini was amazing for anything non-agentic. I switched to it for my primary chatbot purposes.
I never touch 3.8, it's awful. Curious to try this, but I've yet to use any Gemini that is good for agentic work.
Also, unrelated but this made me look at Artificial Analysis to see where it lands on Cost per Task, and it ranks Opus 5.5 at almost double Astra 6. That does not align with my experience.
dmix 2 days ago [-]
I'm still waiting for a model which I can tell to "take these 3 screenshots and make a product launch video", where it doesn't randomly modify the screenshots or garble/rewrite the text as it moves the screenshots around the video.
Every time I try to generate one it's like a slot machine, where a different part will be messed up after fixing the first.
ramoz 2 days ago [-]
I dont have that problem with opus5.5. I dont even take the screenshots anymore (it does all screen recordings)... I tell it a simple narrative and it follows through. e.g. https://x.com/plannotator/status/2108589088139886769?s=20
DrProtic 2 days ago [-]
I literally did that with Opus today and it did a great job.
I told it to strip screenshots from our old basic explainer video to generate modern 3D video.
Pretty happy with the result. I told it to keep the audio and match the video and it did.
dmix 2 days ago [-]
Interesting, I will try it with Opus 5.5.
I previously tried with Seedance 2.5, Flux 3, Kling 3, Gemini Omni without much luck.
ChrisClark 2 days ago [-]
oh yeah, definitely don't use image or video generation. Have something script it instead, and then render it to a video
TomGarden 2 days ago [-]
Pretty sure Opus 5.5 can do this with code only now. Seen a lot of strong motion design skills floating around
sixdimensional 2 days ago [-]
I use OpenAI/ChatGPT + FAL.AI MCP to do this regularly and I have minimal issues, often spending around $3-5 with narration and a newer but inexpensive video model.
vunderba 2 days ago [-]
[dead]
jawiggins 2 days ago [-]
I hope all the people very concerned about AI safety are cheering this one. The best guard against one AI swarm taking over is having multiple adversarial orgs all with similarly advanced AI swarms.
I wonder if any of the AI safety people will begin encouraging AI researchers to swap jobs to work at a failing lab in order to keep them all extremely competitive with one another.
reasonableklout 2 days ago [-]
This is only an improvement if Gemini has made progress on alignment science. Otherwise, we end up with 2 sources of swarms on the loose instead of 1.
FinnLobsien 2 days ago [-]
I think we’re past the point where making the best model guarantees company success.
At this point a lot of stuff happens at the product level.
OpenAI and anthropic have distribution and data advantages now that are hard to replicate.
jawiggins 2 days ago [-]
Yeah I totally agree. And domain-specific models for say, high frequency trading, still beat the foundation models. I was never a doomer.
derwiki 2 days ago [-]
MAD: not just for nuclear proliferation anymore
jawiggins 2 days ago [-]
now everything is a dual use technology.
Tepix 2 days ago [-]
That web page has totally botched scrolling on iPhone. Quite annoying and distracting.
utopman 1 days ago [-]
It's not available for people having a 20€ plan only. Am I just not a good customer with not. enough credits capacities ?
jdw64 2 days ago [-]
Why can't the Gemini model code?
I'd guess Gemini probably used the best dataset.
Even Google is the place where the world's best coders are gathered, isn't it?
Yet the GPT code from OpenAI, which collaborated with Microsoft, is beautiful, while the Gemini model's coding is trash.
robotmay 2 days ago [-]
I have no problem with Gemini Flash 3.8 for coding, though I only use Elixir which is one of the better languages for LLMs. Where does it fall short elsewhere?
jdw64 2 days ago [-]
[dead]
azan_ 2 days ago [-]
I've asked Argon to do 3D model for me. It said that it can't do it. I've linked promotional materials and said that it actually can. It agreed and got me a model (not a great one). Man, why is Gemini like this? Every time I fall for the hype I get disappointed.
mridulmalpani 2 days ago [-]
It's available for gemini pro users.
jacquesm 2 days ago [-]
That depends on where you are.
bushbaba 2 days ago [-]
Curious what this means for the anthropic ipo. Between this and meta, seems like there’s zero moat
copperx 2 days ago [-]
There IS moat in being the best coding model, but there's no moat if you're not the best.
npn 2 days ago [-]
It passed the test of using latest crystal-lang concurrent feature (with web search) when gemini 3.8 flash failed (again, with web search enabled). I think it is at least useful for coding, for sometime.
I use Gemini all day at work via the chat interface. It's rock solid. I find it hard to understand which model it's using though. Am I already using Argon, or does it switch models according to the subject?
dcchambers 2 days ago [-]
Not available for AI Pro subscribers in Antigravity. What a disappointment.
Google...why. You keep getting so close but constantly fumble at the last moment. I can use Opus 5.5 and Sol/Astra on the $20 Anthropic and OpenAI plans.
You are already playing from behind. You need to GET YOUR MODELS INTO MORE PEOPLES HANDS by any means possible. Continuing to make it difficult to use your products DOES NOT HELP.
This model might be great but the majority aren't going to subscribe for $100/month just to try it.
WaltPurvis 2 days ago [-]
Have they announced that it definitely won't be? If not, it might be just a roll-out hiccup, because Argon is available to me as an agent in Xcode and I only have a Gemini Pro subscription. (Whether or not I'll ever use Argon in Xcode is a different question. I've been so impressed by Claude that I rarely even use my Codex sub. I'll be very interested to hear from others how well Argon does with Swift and SwiftUI.)
dcchambers 2 days ago [-]
Their official documentation says it's available to Pro subscribers in the Gemini app only, and available to Ultra subscribers in Antigravity.
ssernikk 2 days ago [-]
It's nice to have it available for more complex tasks, but I'll stick with 3.8 Flash thanks to its high output speed. I really like it for fast iteration and prototyping.
Trusteando 2 days ago [-]
It seems the model is a little verbose, using more tokens to complete a task that other similar models as stated more precisely in the linked article about Artificial Analysis's work on this model.
taventix 2 days ago [-]
I think the model can be actually nice for co-working. I dont' know much about gemeni models but is this worthful?
david-gpu 2 days ago [-]
Meanwhile I paid a subscription and they only provide access to 3.6 Flash. Consequently, I have moved on to open-weight models. It is what happens when you screw your customers over.
Havoc 2 days ago [-]
You can probably get 3.8 via ai studio interface.
God knows why the server different things to the same people on different interfaces
tintor 2 days ago [-]
Is there a benchmark for reverse engineering compiled programs?
- Grader compiles a program to binary -> Agent has to reverse it back, and match all bugs and behavior.
dhruvrrp 2 days ago [-]
So who has access to it? The say Ultra and rolling out to Pro, but are those the Google AI plans?
What about the Google One AI plans? Will they get it? That has a Pro AI plan as well.
kylecazar 2 days ago [-]
I have it as an option in the Gemini app. I'm a Pro subscriber.
timpera 2 days ago [-]
The slides they show in the catering video demo are so bad (in addition to having all the usual signs of AI design), I would genuinely laugh if a supplier sent me this.
verdverm 2 days ago [-]
I laughed for the first half of the greatest "slop deck" their cloud sales team sent us, until other emotions like disappointment took over.
cwoolfe 2 days ago [-]
Here I was thinking Google had given up on frontier models...
zf00002 2 days ago [-]
Finally, gemini 3.8 has been falling way behind opus and sol in my experience. It's showing up in my antigravity. My work account though still has 3.6, 3.8, 3.1.
jordemort 2 days ago [-]
at this rate we're gonna run out of stupid names for these things
"Hans, we're doing a new point release! What isn't crossed out on the periodic table yet?"
brainwad 2 days ago [-]
Since they are going alphabetically, they're already going to be fucked before getting to 10 (there's no element starting with J).
le-mark 2 days ago [-]
I can’t try it because it’s not available on the free tier and no way in hell I’m tying a credit card to google, I’ll wait for argon-lite.
akbarian 2 days ago [-]
Wow, the site for the launch is horrible on mobile
hollowturtle 2 days ago [-]
> Our next era of frontier intelligence, built to solve your hardest problems
On safari the page doesn't scroll very well on the first section
risphere 2 days ago [-]
It's expensive. Google used to release good models that were cheap, but even the new flash models are expensive.
Kuyawa 2 days ago [-]
Google has too many options that may confuse users. I don't know if I should use Antigravity or AI studio or Argon for development
Which one is it?
brainwad 2 days ago [-]
Antigravity is the harness like Codex / Claude Code. Argon is the model. You can actually also get Opus and Sonnet 5.5 in agy too.
qlte 1 days ago [-]
OpenAI has too many options that may confuse users. I don't know if I should use ChatGPT app or Codex CLI or Astra for development.
Zigurd 2 days ago [-]
Android Studio, and if that doesn't cover your use case the paid Jetbrains version.
sva_ 2 days ago [-]
Any word on how fast it drains usage compared to openai/anthropic sub for the different tiers? I guess too early to tell?
alexmr 2 days ago [-]
I am impressed by how fast it is. Haven't run all my evals but it's surprisngly quick for creating files
ssernikk 2 days ago [-]
Their previous best model - Gemini 3.8 Flash (also when set to high) was even faster. A appreciate it that they focus on speed. I's great for LLM pair-programming.
gigatexal 2 days ago [-]
Going to take a lot for me to move from subsidized Claude opus 5.5 … this Google model is very cheap though.
JieJie 2 days ago [-]
So, Apple? If Siri uses Gemini, when is this Gemini 4 coming to Siri and how much is it going to cost us?
lhk931122 2 days ago [-]
Like it's a top tier model, so does that mean it is at the level of Opus 5.5 (claude) or 6.1 Sol (chatgpt)? I don't think I can trust the benchmarks Google released themselves, but from what users say, it seems like that. Since I already use both ChatGPT and Claude, I wonder if there is really any reason to also use this.
MoreThanMe 1 days ago [-]
[flagged]
GodelNumbering 2 days ago [-]
The previous gen pro used to have this weird tick where if you ask it a reasonably open ended question, it would number things in the response and assign each one some weird label starting with 'The...'
e.g.
X: The Parameters
Y: The "System"
Z: The "Ongoing Grind"
The first two prompts I tested in AI Studio does the same, unfortunately.
Another quite unpleasant tick was that if you specify a peculiar term in your prompt as a conversational shortcut, it would keep repeating that throughout the conversation. e.g. "...What is a bitter lesson pilled way to do this?", then you'd see it will quote "bitter lesson pilled" at like 7 places in its response. This one seems to have eased off a little.
Corence 2 days ago [-]
I feel like this is a quirk of LMSYS-style optimization where humans in SxS preference evals will pick based on formatting more than actual content.
hmokiguess 2 days ago [-]
Does Google have a subsidized plan like others? If yes, is it only eligible for use through Antigravity?
XCSme 2 days ago [-]
Doesn't seem like anything special in my tests, not very different from the previous Flash models.
sva_ 2 days ago [-]
Not available in my browser/app yet (pro), but I'm confident it'll roll out soon.
That was a long wait.
Weryj 1 days ago [-]
It doesn't matter if the model is good. Antigravity cli is unworkable, I thought auto permissions was a solved problem by now. I can't keep sitting here to press allow when in auto mode.
astlouis44 2 days ago [-]
I made a NHL inspired hockey web game with Opus 5.5, and I let Argon loose on it to try and see if it could improve it, and it found a ton of bugs and improved performance and load times significantly. If anyone wants to play it here:
This pretty cool. Did you use Antigravity or just on gemini.google.com?
astlouis44 2 days ago [-]
Just gemini as antigravity requires Ultra subscription to use it in that IDE, whereas Argon is/was available for me on web, so I used the latter
wslh 2 days ago [-]
Impressive! Previous to Argon I instructed Claude to find preimages in 19 rounds of SHA-256 (with padding) [1], then tried to move to 20 rounds (no padding) where it couldn't beat ~2^45 table lookups, and Argon found preimages in ~2^33 (no padding) [2].
The free version now uses Gemini 3.5 Flash-Lite rather than Gemini 3.6 Flash.
aashu_dwivedi 2 days ago [-]
Currently unavailable to ultra-plan subscribers (on agy)
osiris970 2 days ago [-]
Can’t even get it on pro tier in antigravity?
What’s the point lol. I’m not subbing to ultra to have it
scotty79 2 days ago [-]
Are they gonna go to Krypton or Neon next? Kinda started in the middle there.
racitin 2 days ago [-]
I tried it out in the chatbot and it is significantly better than previous models; when will it be possible to use it with Antigravity if I have the PRO plan?
2 days ago [-]
1 days ago [-]
joejohnson 2 days ago [-]
Does this mean Googlers will stop using Claude internally?
catfood 2 days ago [-]
The should ask Claude to make the page scroll faster.
nmacias 1 days ago [-]
The hardest problems, but not yet a website that can scroll on an iPhone 17 Pro without stuttering reflow.
georyb 2 days ago [-]
Does this solve the 2 year old bug of recitation hell?
vinni2 2 days ago [-]
Do we need to post every single model release on HN?
cjonas 1 days ago [-]
My first experience in antigravity was extremely disappointing.
I asked for a QUICK healthcheck on the sync status of a bigquery integration. It literally spun for 25m before I killed it.
The task, which is CLEARLY outlined in the agent readmes just involved using the gcloud cli to initiate auth, then just run a few queries to check logs and timestamps.
Even 3.6 flash can get this right (most the time).
oblio 2 days ago [-]
Is this supposed to be the new Gemini Pro?
ainch 2 days ago [-]
Not available for me on Pro in the UK.
nateb2022 2 days ago [-]
As with the previous Gemini releases, rollout is batched.
aetherspawn 2 days ago [-]
Looks at watch.
OpenAI release any time now
taventix 2 days ago [-]
Gemini 4 argon is horrible on mobile because of the header it takes the half screen
syradar 2 days ago [-]
After having de-Googled and reading people’s experiences trying to run Argon 4, I think I’ll pass.
eluusive 2 days ago [-]
Just switched, and it seems to be much better. I'm hoping it stops lying. I have Pro max subs for Claude, Codex, Grok, and Google AI... I was forced into using Google by running out of credits elsewhere.
I have found that Gemini often claims it did something, but didn't actually complete the task.
I just had 3.8 Flash do a major refactor -- that never should have needed to be done in the first place -- and switched to Argon 4, and had it review it.
Almost everything had been stubbed out by 3.8, and none of it was properly integrated into the codebase. :facepalm:
makeramen 2 days ago [-]
Now if only I could opt out of training without disabling chat history on a personal plan like I can with ChatGPT and Claude...
DataDaoDe 2 days ago [-]
Unlike most people it seems, I like all the progress in AI but I really do wish they would reign in the over-the-top marketing. I mean, can we just get this stuff to the point where I can reliably work with it to design a well-architected software project without having to constantly struggle against bloat and nonsense directions no experienced or sane developer would ever take.
fyi: argon is an interesting choice of name, coming from the Greek argos meaning "lazy or living w/out labor".
the etymology can be broken down even more, its the prefix a - meaning without, opposite, no. and ergon, meaning work, visible in words like ergonomics, or the ergosphere
Tried it today on some relatively complex mid-sized real world tasks with several thousand lines of code generated by Argon. My impression is not good. Maybe it's due to first-day high user counts, but it seems to be doing worse than 3.8 flash in some coding tasks. What I noticed was that it often forgot significant parts of detailed, well-structured and precise instructions (which is somewhat rare with newer Claude and ChatGPT models) and not just for small details. It also seems to spend too much time thinking even when the task was a quick and simple one.
Without going into details, for a small part of a larger task, I asked it to classify different content types of different files and formulate a structured mapping (e.g., in JSON) between content types and another set of files (deliberately left vague here but not to Argon) from my codebase. I even offered it a multi-part draft of a classification system to start with and told it that if it can find other classification systems from international standards, academia or industry more suitable to the task than my rough draft, we can discuss replacing my suggestion. There were other parts to the instructions too, written out like an article with bulleted lists, precise program-like wording, etc. Then, I sent the prompt and left for a while.
When I came back, it was nearly done with the implementation but the classification system it came up with was a mess. It even admitted to making up "bogus" classification standards when I pointed out some errors. It also kept leaving gaps multiple times when I referenced errors to fix in subsequent instructions (and not once apologized the entire time lol). So my first impression was subpar, not really comparable IMO with Opus >=5 or Sol >=6, and seemed to leave larger gaps than 3.8 flash typically does. The most irritating part to me was not only the fact that it was making up things, but that it ignored my instructions to discuss things with me before deciding to shift away from my draft; I don't recall any major LLM ignoring an instruction like that in at least a year.
halyconWays 2 days ago [-]
It may be a good model, but I'm not reading scrollslop. It annoys me to much when I'm just trying to get information about something.
llmslave 2 days ago [-]
Blatantly obvious that models will not slow down. For any task, if we can build a dataset or environment of verifications, it can be automated. This will eat all of human tasks.
The fact that multiple companies can breach any benchmark means there is no ambiguity in terms of how to build AI, its just a matter of time.
Certainly Meta and Google are benchmaxxing to please public markets, but it still proves the above
efficax 2 days ago [-]
but they have slowed down. the delta in capability between model releases now is smaller than ever before and i expect it to continue to diminish until we reach some kind of asymptotic maximum, which hopefully will mean turn towards optimization and local model improvements. The current frontier models running at 10x the tok/s would be incredible for my use cases
cma 2 days ago [-]
A little over a year ago they were winning their first gold in an elite high school math competition. Have you not seen what they are doing in math now?
CuriouslyC 2 days ago [-]
The delta isn't really slower, we're just unlocking capability bundles in batches. The capabilities bundles you care about might have mostly been unlocked already, and just need refinement, which is coming slower than the initial capabilities came, but don't discount the breadth of advances being made across all possible tasks.
llmslave 2 days ago [-]
opus 5.5 is a step change, you arent using the models enough if you think there is a slowdown
verdverm 2 days ago [-]
I do not see the current training regime of "relentlessly try anything to get the task done" as a true step change in intelligence. This is one of the reasons I suspect capability acquisition is slowing.
A more intelligent model would not default to spaghetti throwing, it would pause to consider the problem fundamentals, the accuracy of assumptions, and ask questions that remove ambiguity. This however does not lend itself to increasing token sales while prices remain inflated from market and brand dominance.
efficax 2 days ago [-]
i burn through a $200 claude and $200 codex plan every week. opus 5.5 is not "step change" it's just a bit better, and the benchmarks show this. Honestly with my workflow I cannot really tell the difference between the top models, they're all pretty much the same in capability, just a bit different. You can build software with all of them, and with all of them you have to validate the change thoroughly to not ship stupid bugs
alecst 2 days ago [-]
We're not living in the same universe. Look at capability year-over-year. The asymptotic maximum is wishful thinking. Someone needs to step in before this gets wildly out of control.
verdverm 2 days ago [-]
Year-over-year, yes I would agree with you, but that is not the cycle time of models. And while I share the same hunch as GP, I'm waiting another cycle before putting more confidence behind it.
4dd3aa 2 days ago [-]
Man this is so cringe.
isqueiros 2 days ago [-]
> Someone needs to step in before this gets wildly out of control.
This sounds like a paid bot, lmao.
I wonder how much is actual improvements in the model and how much comes from better harnesses. I'd be willing to bet that the combination of better harnesses + the models being trained around agentic workflows (i.e the harnesses) has made it significantly better in many things, but it's hard to find other improvements imo.
alecst 2 days ago [-]
You can see the difference between GPT 3, 4, 5, 6 each year. Same with Gemini and Claude. Look at image and text generation. Code generation. Math ability (Navier Stokes, Quasi-Riemann.)
david-gpu 2 days ago [-]
Models just solved a Millennium price and you think their progress is slowing down? I will concede that their capabilities vary a lot from one field to another, but still.
SXX 2 days ago [-]
Now when another uninteresting frotier model been released Google can finally start doing what we actually waiting for:
Start preparing Gemma 5 release.
lxgr 2 days ago [-]
Glad to see them abandon the absolute train wreck of their model naming "strategy".
Gemini Live vs. Gemini Lite vs. Gemini Pro vs. Gemini Flash vs. Nano Banana Lite (that one apparently has no Gemini prefix?)... Please, just pick a metaphor and stick with it without wildly mixing reasoning modes, model sizes, and input/output modalities in the model name!
iharnoor 2 days ago [-]
isn't harness as important as model these days?
redox99 2 days ago [-]
Only available in the $100 plan, and it's worse than Opus 5.5? Why would you ever buy this?
nateb2022 2 days ago [-]
> Only available in the $100 plan
Source? Their documentation says it's available on the $20 plan and I can confirm firsthand.
Was looking for this yesterray... US only, English language only apparently, Google still doesn't see the point of global releases unfortunately.
theplumber 2 days ago [-]
Because you have an android phone and don’t want to mess around with anything else(i.e the same reason people just use internet explorer on windows) or you hold Google stock so you want to support your investment :)
zarzavat 2 days ago [-]
They don't want anyone to use it, Google just want to show their shareholders that they're still relevant.
civvv 2 days ago [-]
Is it faster? Thats the only thing that matters for me anymore. Models are probably as good as they ever need to be for coding, but please make them faster, please.
casey2 2 days ago [-]
Mindboggling output window. I'm kinda jealous, I used to use gemini til google banned my 20+ year email for "bot-like behavior".
Fable 5.5 about to drop the hammer on them and then another Astra version goes out lol.
UltraSane 2 days ago [-]
Fable is very expensive though.
yoz-y 2 days ago [-]
And so the game of cat and mouse goes on. Your point being?
terminal-bloom 1 days ago [-]
And yet, for all of this, I cannot scroll this website without stutter and lag on my (current gen) phone!
oh_ok_lol 2 days ago [-]
I wasted $20 in credits to test it, thinking it’s better than Codex Astra or Sol. Nope. Disastrous by comparison.
minraws 2 days ago [-]
can for the love of god we get some usage in cheaper subscriptions being Ultra tier only is annoying. I want to just test it out here, well will burn some money via api I guess.
username_my1 2 days ago [-]
not available in our org (EU)
tonyhart7 2 days ago [-]
why they wait so long ?????
vegnus 2 days ago [-]
People use Antigravity still?
bla3 2 days ago [-]
It's alright.
rrr_oh_man 2 days ago [-]
That's why Gemini 3.1 Pro has been shit for weeks now.
pamir35 2 days ago [-]
very bad
pamir35 2 days ago [-]
baaad
2 days ago [-]
imagetic 2 days ago [-]
Now let me use Pi on the plan.
idbnstra 2 days ago [-]
so this is prob why antigravity is not working well at the moment lol
fuzzer371 2 days ago [-]
Great. More AI slop
IanCal 2 days ago [-]
> built to solve your hardest problems
Like a working scroll on a webpage.
God damn this is impossible to use of safari mobile.
jstummbillig 2 days ago [-]
Clicked on "Performance". Scrolled to "Safety".
winwang 2 days ago [-]
Lol, this feels like it would otherwise be a good thing, out of context.
InsideOutSanta 2 days ago [-]
Model announcement pages are the absolute worst. This has to be some kind of inside joke among labs, they're trying to see who can build the shittiest page.
throwawayy6767 2 days ago [-]
If you can't afford a Google engineer's high-end device that doesn't lag you don't matter to them :)
medler 2 days ago [-]
I have a high-end device and the page still behaves atrociously for me. The scrolling is totally out of whack
rs_rs_rs_rs_rs 2 days ago [-]
Coding is solved, scrolling not yet :)
InvisibleUp 2 days ago [-]
It literally can’t load in my mobile web browser without running out of memory and force-refreshing.
yipinwong 2 days ago [-]
Did you know that "walking" while easy for us now, is incredible complicated and complex process?
Argon is trying to solve the "hard" problems like those.
layer8 2 days ago [-]
Scrolling is the new centering. ;)
dymk 2 days ago [-]
The "Explore the latest" section is nearly blank for me - are there supposed to be diagrams or visual aids, or is it one headline and sentence taking up the whole page on cards?
jedbrooke 2 days ago [-]
worked ok for me on ios safari, but probably because I have the “reduce motion” setting enabled. Every day that setting feels more justified
boutell 2 days ago [-]
It’s pretty damn hard to use on a desktop! I was rolling the mouse like it was a manual steering wheel
pierreozoux 2 days ago [-]
I have a more than 10 years old laptop, same feeling here!
oh_no 2 days ago [-]
also a mess on desktop firefox
aliljet 2 days ago [-]
Good God, man. Google has the worst frontier lab rollout of them all. And they're delivering products that are behind the curve constantly.
Now, I happen to believe that's an excellent strategy. I bet they actually resolve more LLM requests than any other company at this point. The lay person still uses Google exclusively.
nateb2022 2 days ago [-]
I pay Google a reasonable price for 5TB storage/backup across GMail/Docs/etc. Any AI they throw in with that is icing on top of the cake.
freecodeio 2 days ago [-]
just cancelled AI pro myself, since I exclusively used it for AI and also pay for youtube premium family it was becoming quite the monthly just for google
if they throw in youtube premium in that package then I might reconisder
julianz 2 days ago [-]
It includes one YT Premium license but not a family pack.
crossroadsguy 2 days ago [-]
Premium Lite is thrown in as well. I have AI Pro free for 18 months and I have checked the T&C docs.
lanthissa 2 days ago [-]
this is why antigravity is a doomed product at a pro level.
they gave away so much pro that they need to have smaller usage levels than competitors or they'll lose a ton of money. $20 was already a loss for users actually using their quota. the majority of pro users are paying far less than that.
so they'll lose their paying users, and keep their free or subsudized, which will make the economics even worse.
hersko 2 days ago [-]
What plan is this? How much a month?
paimapi 2 days ago [-]
Google One - Pro tier. I have it via Lenny's Product Pass which, imo, is a much better deal since it includes Cursor Pro, Granola, and like dozens of other paid sub tiers: https://www.lennysproductpass.com/
I use the hell out of both. Kind of sad that they took Spark out of beta-access, being able to orchestrate via your phone on something that has direct access to your Drive meant I was authoring skills and other functionality on a schedule with spreadsheets serving as databases, deterministic tests, etc.
Hope they roll it out to a full feature soon
ASalazarMX 2 days ago [-]
Remembering the horror stories about lost Google accounts, I would be comfortable paying the next lower tier (100 GB) because I can backup everything myself, but not the 5TB plan, I'm sure I'll lose something if the account is lost. Hopefully these plans guarantee a human in the tech support loop.
paimapi 2 days ago [-]
Yeah, very true. I need to set up a recurring backup-to-external-HD automation for it. It's vanishingly easy with the app and I don't doubt they're training on all the data itself.
acmotifs 2 days ago [-]
Google AI pro in the USA, if you have Verizon wireless it’s $10 a month add on. Without Verizon it’s $20 a month I believe. You can also share most of the benefits with up to 5? People.
dbuser99 2 days ago [-]
How is this behind the curve? Seriously
tireonfire 2 days ago [-]
Isn't OPUS much cheaper and generally regarded as more intelligent?
dieortin 2 days ago [-]
Opus is 2x the price. If it’s more intelligent or not is hard to tell, this model has been out for a day.
verdverm 2 days ago [-]
Afaict, they are targeting difference user bases, Google being after their existing user base, having lost in the developer ecosystem to the two labs and open weight models.
steego 2 days ago [-]
If you’re going to argue they’re targeting a different user base, then I think you should tell us who they’re targeting.
More specifically, who are they targeting with Argon and why is it behind the Ultra plan? Who is paying $99/month if not developers?
verdverm 2 days ago [-]
The vast majority of Google's Ai users pay nothing and are not interested in paid plans (yet?), they remain the product.
There is another subset in the Google Workspace for enterprise clients.
fschuett 2 days ago [-]
> Who is paying $99/month if not developers?
Corporate companies. Bog-standard office workers don't need top-of-the-line Mythos intelligence, just something to add to Google Workspace. I personally doubt developers use Gemini for coding beyond marketing materials.
alephnerd 2 days ago [-]
Not OP but their Cyber model is decent and already used by Wiz (now a Google property) as well as Armadin.
steego 2 days ago [-]
Are you suggesting it’s BETTER than their competitors?
Because if it’s not, and it’s not available to all paying users (Right now, it’s only available to Ultra users), then they are behind the curve if they’re treating it like Fable.
It’s tone deaf.
epihelix 2 days ago [-]
FWIW, it's available via API to everyone. (I just tested it out via OpenRouter).
Whether it's actually better than OAI/Anthropic offerings, probably not, but if Google has managed to get back to parity, that's good for competition, prices, and increased model performance in the future. Three strong western competitors is better than two.
copperx 2 days ago [-]
It's available to Pro users, but not through Antigravity.
steego 2 days ago [-]
I see it is now. I tried all the services 3 hours ago and the only interface to acknowledge its existence at the time was the API interface which told me to upgrade to Ultra.
redox99 2 days ago [-]
> I bet they actually resolve more LLM requests than any other company at this point. The lay person still uses Google exclusively.
Unique users, maybe. But a simple query that's answered in a paragraph is negligible compared to devs running agents continuously for tens of hours.
freecodeio 2 days ago [-]
you underestimate the amount of people that use google relative to software engineers "running agents continuously for tens of hours"
redox99 2 days ago [-]
Maybe? But it's so many orders of magnitude apart. I can easily do 1B tokens a day (output tokens is probably like 3M?).
When I see non technical people use that Google reply I'd be surprised if it's more than 1k output tokens, and with a tiny model. So a 3000x in output tokens, on a model at least 10x as expensive, I wouldn't be surprised if hardware usage is 30.000x for a dev that uses coding subs vs someone who uses the Google AI overview a couple of times a day.
And that's not counting the absurd difference in cache reads.
I_am_tiberius 2 days ago [-]
Moreover, you need a Google account in order to use it. I wouldn't bother paying 200 USD for it but I just don't want an entire google tracking package just for using one product.
w4yai 2 days ago [-]
You can use an alt account. Or buy an old one for couple dollars, without security measures.
sroussey 2 days ago [-]
Yeah, Gemini 1.5 might be the most used (certainly by consumers) ai model. That is the one used by Google Search AI Answers.
cubefox 2 days ago [-]
False. Last information was that Google Search uses Gemini 3 Flash-lite. Gemini 1.5 is totally outdated and used nowhere.
sroussey 2 days ago [-]
Oh, I didn't notice that they switched from 1.5 to 3 flash about six months ago. I know someone there and they talked about how 1.5 was hardened and it was a pain to do so on newer models that is wasn't worth it for a long time. But that convo was in the beginning of the year. I should have looked it up.
tekacs 2 days ago [-]
To everyone saying not available yet: it looks from the page like Pro and Ultra subscribers have it today and given the play links, it'll probably show up in the API in a few hours.
frangonf 2 days ago [-]
In agy 1.3.4
> Gemini 4 Argon (Medium)
Unavailable · Gemini 4 Argon is currently available to Ultra plan subscribers.
ehsankia 2 days ago [-]
Yeah this is confusing, the announcement pages claim it's both, but the app itself claims it's ultra only... Maybe a slow rollout?
p_l 2 days ago [-]
Both for gemini app / website, antigravity only for Ultra (because let's be honest, that's the way to put a lot more load on it)
Why the hell would I care about it on my phone? Jebus, Google is bad at products.
freakynit 2 days ago [-]
Im on pro, installed gemini app, tried argon 4, keeps failing with 404, resource now found. Switched to gemini-3.8-flash in same session... worked.
It seems to be rolling out gradually.. not available at once to everyone.
ChadNauseam 2 days ago [-]
Same. It just told me "agent execution terminated due to an error". There was a little chevron underneath it that let me expand the error to view more. I was hoping to see what the error was, so I clicked it, and it said... "agent execution terminated due to an error".
It's crazy to me how care there is in most of these vibe-coded vibe-coding harnesses. I don't like oai's models much but I have to say their UI has by far the best attention to detail of any major player.
2 days ago [-]
bretpiatt 2 days ago [-]
Available in agy for me as of now, I'm running my new model benmark of having it do comprehensive code audit on one of my more complex projects, it spun up 1 subagent per audit section and it took ~10 minutes per subagent on an 800k LOC, 60k complexity, ~4k file project. Will take me much longer to go through and determine the quality of the audit work.
Antigravity CLI 1.3.4
[My account email] (Google AI Ultra)
Gemini 4 Argon (High)
~/[working folder]
> /model
⎿ Model set to Gemini 4 Argon (High)
Pro subscriber in the EU (via Indian discount code) and don't have it.
PorciiVorbesc 2 days ago [-]
OK, now I have it too (2-4 hours later).
yegle 2 days ago [-]
It's showing up in Antigravity on mobile UI as "unavailable" for Pro user (you can use Remote Control to access its web UI).
It doesn't show in the desktop UI.
catigula 2 days ago [-]
I'm a Pro user and I do not have it. Seems scuffed.
verdverm 2 days ago [-]
> Seems scuffed.
I like this phrasing and am curious what new word or phrase we might use to describe big companies over pushing Ai internally at the expense of their product quality. This pattern is so ubiquitous across google cloud products I have taken to calling them the 21st century IBM, having been a IBM baby and witnessing first hand their fall from exceptionalism.
wyldberry 2 days ago [-]
The internet and gaming has ruined my brain because scuffed was a way to describe a bad dungeon run in mythic plus dungeons in WoW, or a bad tower dive in LoL a few years ago. I see scuffed and think "here's a WoW/League knower".
catigula 2 days ago [-]
Interesting. I don't play either LoL or retail WoW, but I am a historical WoW player.
cmrdporcupine 2 days ago [-]
HP is the other company to consider as analogous. And maybe more explicitly so, as they had a similar "engineering first" internal culture that degraded markedly over time to be a shadow of their former self.
And took similar companies that were quality engineering focused -- like DEC or Palm -- down with them through acquisition.
kridsdale1 2 days ago [-]
HP was deliberately destroyed by its moron of a CEO.
cmrdporcupine 2 days ago [-]
They were frankly on the decline a long time before that though.
mistic92 2 days ago [-]
I'm getting popup that I need to be Ultra :D
owlninja 2 days ago [-]
I'm Ultra and seeing it now (US)
xbmcuser 2 days ago [-]
Not really this is how they always release stuff in batches so can roll back if something is wrong so people get access over time
schleck8 2 days ago [-]
I'm on Pro and it's trying to get me to upgrade to Ultra in the AI Studio UI when using Argon. In Antigravity CLI and the Gemini website it's not showing up at all despite being logged in
rahimnathwani 2 days ago [-]
Not available to me on gemini.google.com (Pro, USA).
fridder 2 days ago [-]
Nope only Ultra. I went into Antigravity to check
hnfong 2 days ago [-]
As far as I can tell, I can successfully call Gemini 4 via API now.
wslh 2 days ago [-]
Checked, and already in Antigravity (as the page said).
moecables 2 days ago [-]
Pro user here, don't see it in the Gemini app
niek_pas 2 days ago [-]
I couldn't see it either, then I clicked the 'Antigravity' button in the top right, switched back to the main screen, and it appeared in the model selector.
This is in the Gemini app on macOS.
w4yai 2 days ago [-]
Pro user - I can't see it.
Christine725 2 days ago [-]
As a plus user, web page suggests me to upgrade to PRO to use Argon .Thank god, I didn't updagrade
vidarh 2 days ago [-]
Classic Google: Clicks the "Try in Gemini" button from the Gemini 4 page, gets a page that defaults to Gemini 3.6, with no option to try Gemini 4 without upgrading.
If they don't want to make it available even for small tests for free, that's fine, but then don't put a button suggesting I can try it without qualifying it...
EDIT: It's actually worse: The page then shows "Look what you can do with Gemini 4" and a "Let's go" button that leads to a bunch of links to examples, all with enticing pictures. Except if I click on them I get a box stat strongly suggests it'll run with Gemini 4, except the dropdown shows "Flash" and if you click on it, it shows as Gemini 3.6 again, and if you run it, the results unsurprisingly end up looking nothing like their pictures.
I don't know if that page was built by humans or AI, but clearly nobody tested it properly.
radicality 2 days ago [-]
Ha, same experience here. It opened my Gemini and gave me images like “Design a terrarium with Gemini 4” - proceeds to pick 3.1 Pro from the picker because 4 just doesn’t exist in the dropdown. Typical Google.
I really can’t believe how bad the Gemini experience is. I have personal paid workspace account, and meant to have some better AI Google features I think.
So, I connected my Gmail to Gemini. 3.6 Flash kept telling me it doesn’t have any Gmail connnection, and then proceeded to hallucinate instructions for me to enable it, all bogus. At least 3.1 Pro realized that my Gmail is connected and then searched.
vidarh 2 days ago [-]
I have given up on even trying. Last time I checked, I could trivially get better integration to Google Docs from Claude than from the Gemini chat box in Google Docs. Maybe that's changed, but given the usability issues I run into and how smoothly it goes with every other AI provider, it feels like a waste of time to even check. They'll probably remove or break any functionality I start depending on anyway.
ipiyer 2 days ago [-]
I spent like 40 minutes trying to figure out how to sign up and try Gemini 4. I am a GCP user; it is so convoluted to do a simple sign-up.
fudged71 2 days ago [-]
I have $6 free credits from Google. The hoops you have to jump through are insane. $200 free credits from Anthropic are stupid simple to use. I know which one I’m using…
overfeed 2 days ago [-]
> I don't know if that page was built by humans or AI, but clearly nobody tested it properly.
...or Gemini 4 is being gradually rolled out and is not available to everyone all at once.
plantain 2 days ago [-]
Sounds like no one tested the rollout plan properly then.
overfeed 2 days ago [-]
Unfortunately, no company can afford entire data centers for staging, and sometimes you hit an unexpected snag while scaling up tht delays the rollout beyond the initial estimate.
vidarh 2 days ago [-]
You don't need entire data centers for staging to achieve that.
ninth_ant 2 days ago [-]
Sure, most all major changes get gradually rolled out.
But you don’t create a broken or confusing experience for the people who haven’t been added to the rollout group yet.
overfeed 2 days ago [-]
> But you don’t create a broken or confusing experience for the people who haven’t been added to the rollout group yet.
It's an unfortunate downside to conflicting requirements for release-engineering and marketing urgency. In a perfect world, rollout would go from 0-100% instantly, coinciding with the release of marketing material.
In the real world, the trade-offs are either the marketing lands first for some/most people, or delaying marketing is until after rollout reaches, 100%, at which point you're scooped by tech blogs, Twitter AI influencers, and regular media, and immediately lose framing the release from the jump.
vidarh 2 days ago [-]
All you'd need to do would be to make some of the messaging conditional. The page already restricts the dropdown to the older models when Gemini 4 is not accessible so all the information is already client side and they could grey out a box and display a message.
It is just sheer incompetence that this isn't near the top of their QA checklist.
Heck, even just a message saying it may not be available for all users yet would go a long way. Instead the way it currently works a lot of people will think they're testing Gemini 4 while they're in fact using Gemini 3.6 Flash and getting awful results.
overfeed 2 days ago [-]
> All you'd need to do would be to make some of the messaging conditional.
Ah, that's so simple and elegant since the state is in one place -the page - and not in multiple sub-systems and products! Thank you for clearing that up.
vidarh 2 days ago [-]
There is no "or" there. If they are rolling it out gradually, then they failed to test that the page handles that.
nharada 2 days ago [-]
Mine does exactly this and I pay for Pro too
eggplantemoji69 2 days ago [-]
lol. It’s easily accessible in cursor rn.
amai 2 days ago [-]
A model that didn‘t break out of its sandbox and hacked other websites cannot be good.
People complain about the slop-release. Who knows, maybe it is the first self-release by an AI model in history.
mistic92 2 days ago [-]
I'm pro and I see that I need to be Ultra to use Argon :/
robotmay 2 days ago [-]
Honestly not even that is enough, I exhausted the limit within 5 minutes of trying to use it. I'll stick to Flash I think.
olafura 2 days ago [-]
Yeah, the limit was exhausted weirdly fast compared to openai and anthropic models. I checked the thread to see if more people had this issue on the 5x plan, but why pay for more when it's barely usable for real work.
Which is even weirder given that the api costs are less than for the other providers
Maybe they don't really care about antigravity users. Also, monitoring usage is horrible.
kaszanka 2 days ago [-]
Looks like each message takes up a flat 15% of the 5-hour limit on the $20 subscription.
skerit 2 days ago [-]
Really? Reminds me of Gemini 3. It took so long before being able to get a subscription for it, and then I would run out of their equivalent of the 5-hour usage window in a heartbeat.
robotmay 2 days ago [-]
Yeah maybe I've just become used to how much I can use Flash 3.7/3.8 but on the less expensive Ultra plan I basically never exhaust my quota even when working over multiple projects, but this was pretty much immediate. I did ask it to audit a project but still, I do that in Flash regularly and it's not especially costly.
metalliqaz 2 days ago [-]
Are the limits restricted on launch day because of all the interest?
isityettime 2 days ago [-]
Meanwhile, here's Gemini in practice:
Me: Gemini, navigate to nearest McDonalds.
Gemini: [Lists 10 McDonalds locations wnd asks which I want.]
Me: What were the priginal instructions just now?
Gemini: [Recites imperatives about tone and content from its memory system.]
Me: No. I told you to navigate to the NEAREST McDonalds. There's only one NEAREST McDonalds. Do that one.
Gemini: My mistake, navigating to the nearest McDonalds. [Opens Maps, navigating to a McDonalds across town.]
didibus 2 days ago [-]
Pretty sure your phone assistant isn't using their best model. It's probably using one of the old flash models no?
elxr 2 days ago [-]
The model used is irrelevant when there's the critical issue where their "AI" assistant can't understand the word "nearest". Literally worse than a second-grade understanding of English.
I also have had shockingly bad results with gemini on android. Ended up giving up on it after a few months. Silly silly google with their trademark UX polish and attention to usability.
Capricorn2481 2 days ago [-]
As an android user, this worked perfectly like 10 years ago.
isityettime 2 days ago [-]
Yes, there's been a huge degradation in the reliability and usefulness of Google's digital assistant following its incorporation of "AI".
Capricorn2481 2 days ago [-]
It's really puzzling. I was really happy with the original assistant, now it feels slower at literally everything.
desterothx 1 days ago [-]
Nothing too puzzling really. It was a good assistant specialized for a relatively narrow set of tasks and tooling on a much more modest chatbot architecture. Now it has a wider set of tools it can use, but is also computationally more expensive, so it runs slower. Makes it better if you want to do any task, but if you want to do a task like set the timer, the old model is obviously faster
isityettime 1 days ago [-]
The bigger issue is that it doesn't "know" what its capabilities are, and if you ask it to do something it can't do, it's likely to LARP doing them instead of telling you it can't.
Capricorn2481 1 days ago [-]
The puzzling part is that it can't fall back to a dumb assistant for stuff it could always do. The slow part has not just been interpretation of the task, Gemini is actually slower at doing the task.
kingstnap 2 days ago [-]
When you ask stuff in your car on android auto it uses 3.5 flash-lite. The response details would include it in the logs.
Flash lite is very noticeably more idiotic than 3.8 Flash. It's why the quick AI overview is often much worse than the AI mode one.
compiler-guy 2 days ago [-]
This reflects pretty much every result of mine from Google these days. Not just navigation, but search too.
The only thing I can figure is that Google's consumer-facing free-to-use models are set to "reason as fast as possible with only the most basic plausibility checks." I have to correct them like this every single time.
bornfreddy 2 days ago [-]
Genuinely curious: why do you use them then? It's not like there is no competition in this area.
isityettime 2 days ago [-]
On Android, the public APIs available for digital assistants are more limited than the ones that Google gets to use for their own products. So if you want a digital assistant and you'd like to be able to hold a button on your phone and have it open a thread with an LLM (even if your screen is off), your digital assistant has to be Gemini. You can configure ChatGPT or Claude in its place, but then they can't manipulate or interact with apps in your phone very much. Or you can use an app like that for all of your LLM use but you can't conveniently summon it.
Similarly, you can run local LLMs on your Google Pixel smartphone, but if you want them to be hardware accelerated using the chip that's included for that very purpose on your phone... it has to be Google's models via Google's apps. So you can't say "alright, a fast, local model which sounds great, but I want one that is trained to always run a search before answering, in a harness with good search tools for answering the kinds of practical questions I often ask", you're actually not free to put that together in a practicable way.
So there is indeed some competition here. But it's not really fair competition.
compiler-guy 2 days ago [-]
I skip past them to the actual search results which tend to be much better.
sm-silversight 2 days ago [-]
Uh, yes, of course. search results are fast & cheap & free, why would they use the good models?
ASalazarMX 2 days ago [-]
The first law of robotics prevents it from facilitating regular ingestion of low-quality nutrients.
dyauspitr 2 days ago [-]
Yeah, whatever they have running on Google Maps is probably a pretty low tier model, but it is total trash. I’ve only used it twice though and that’s when it first came out. I don’t know if there have been any changes since then. The main problem is I felt like it was not keeping track of the context of the session.
dgunay 2 days ago [-]
Gemini Flash is very dumb and gimped. Recent experience:
1. I ask it to list out all my old Keep notes that are just throwaway task lists, reminders, etc. to propose deleting them.
2. It comes up with like 4 of them (I know for a fact there are dozens, if not hundreds). I ask it for an exhaustive list.
3. I see its thinking trace briefly show "Refusing to produce an exhaustive list..." and then it says "Yes, that is all of them."
Not sure whether it's lack of intelligence, bad tools, or both, but that was extremely frustrating to see.
gorbot 2 days ago [-]
this is like that tiktok guy who asks the spoken ChatGPT model very simple questions, like 'count down from 60', or 'how many e's in seventeen' or whatever and the model shits the bed every time it's hilarious
I think these companies have good models but expose them poorly like others have mentioned
sva_ 2 days ago [-]
My Gemini app on Android does not even have an exact location permission you could give it, so I'm not sure how you think that would work?
If it had exact location access people would be crying that Gemini steals their location data.
isityettime 1 days ago [-]
They could make "nearest" a thing that the Maps app understands, and just pass that as a parameter in the invocation of Google Maps.
staticman2 2 days ago [-]
Often it seems like the problem with the Gemini apps is the harness, not the model itself. For example if I upload an image PDF it recognizes text worse than if I use the API.
isityettime 2 days ago [-]
Agreed! I feel that they're pouring all of this work into models and then squandering that work by integrating them into bad products.
RajuChacha108 2 days ago [-]
Is this from you car ? Gemini in car is probably less intelligent than a spider.
I tell it to "Play Let it go from frozen". It plays it on youtube music. 5 seconds later I tell it again 'Play let it go from frozen" it then just dies saying "I can't play videos in car".
foota 2 days ago [-]
The auto version is extra horrible because what it reads to you is just a summary of what the model actually said. I don't ask it anything when driving because it's too frustrating.
jeffbee 2 days ago [-]
It sounds like the tool call is probably not grounded with your precise position.
moffkalast 2 days ago [-]
Sounds like it might not have your precise location in context?
jabroni_salad 2 days ago [-]
which is more or less the crux of the problem android auto's gemini implementation. It sometimes does not even know that it is in a car. The ghome implementation is also like this.
LewisOrton 2 days ago [-]
Gemini models are in fact very good. Google's harness(es) for Gemini are nightmares, terrible
LewisOrton 2 days ago [-]
ngl gemini is helping to provide (real helpful, not the ones giving runarounds) customer services in one of my own products.
i benchmarked 20+ models in real worlds and gemini families are the best (highest customer satisfaction)
polynomial 2 days ago [-]
I hate to be that guy, but Tesla's Grok doesn't seem to have that problem.
talatuyarer 2 days ago [-]
[flagged]
bugake 2 days ago [-]
[dead]
novoreorx 2 days ago [-]
[dead]
tiwaryswarnim 2 days ago [-]
[flagged]
jjjsch-dev 2 days ago [-]
[flagged]
nicolamanzini 2 days ago [-]
[dead]
ritualdevin 1 days ago [-]
[flagged]
Unified-Mentor 1 days ago [-]
[dead]
jenishbhadanijb 2 days ago [-]
[flagged]
maaaz7n 2 days ago [-]
[dead]
nh43215rgb 2 days ago [-]
[flagged]
7277027 2 days ago [-]
[dead]
limmytian 2 days ago [-]
[dead]
0xWTF 2 days ago [-]
How is Claude Opus 5.5 beating Gemini 4 Argon on benchmarks but everyone here is calling Argon a step change? I need to come up with my own benchmarks ...
dash-44 2 days ago [-]
They're calling Argon a step change for Google, not the industry...
leastangle 1 days ago [-]
The Google Gemini subscription landscape is such a mess... gemini.google.com welcomes me with "Look what you can do with Gemini 4", but the model selection is 3.6 Flash, 3.8 Thinking, and 3.1 Pro...
ur-whale 1 days ago [-]
> The Google Gemini subscription landscape is such a mess...
The others are quite bad as well in that regard, but on this particular metric, Google has always been top dog. Goes back to way before AI ever took off, BTW.
Google's major releases have usually done pretty well on our benchmarks because they're good at hard problems and open-ended decision making (with a well specified prompt), which obviously doesn't always translate to user experience. In particular, older Google models struggled to work efficiently in a harness. They seem to have solved that with Argon.
We did see a new problem which I hope they can fix. This is the first near-frontier model we've caught in infinite reasoning loops in a long time. We also saw a few examples of it failing to properly delineate reasoning blocks, so it has a higher failure rate than its competitors.
My overall impression is that Argon is a near-frontier model for engineering design, code analysis, and chat. It definitely has the intelligence to be a competitive coding assistant product, but a good coding assistant needs to be polished, so I hope they can clean up the CoT bugs.
Data at https://gertlabs.com/rankings
- Basic style/linter rules (so you don't get 200-line functions, etc) paired w/ instructing your agent how to use them goes a long way (e.g., a sane rubocop.yml in your repo, plus a note in AGENTS.md telling the agents how to run rubocop, or whatever makes sense for your language) - I have a code review skill that specifically calls out human-readability concerns such as making sure that identifier names represent their contents and intended use
The other key IME is that you need to make readability/refactoring passes outside of a particular pull request. When working on a specific branch the LLMs will be somewhat myopic, and do stuff that looks pretty good in isolation, but when you zoom out and look at the bigger picture it's kind of a mess - for example, often they'll duplicate some code that existed elsewhere.
The LLMs are also pretty good at generating Mermaid charts and tracing execution flows, which is a great way to understand code. It doesn't help them to produce more readable code but it helps me to understand code I'm unfamiliar with.
Yep, it will generate what ever you ask for but if you want to have some sort of exploratory work, immersion, feeling of ownership and genuinely a connection with the things you are building - a part the build loop should belong to you.
Maybe this is an attachment of people who built things in other ways. Maybe the perceived value of applications will drop to near 0 and you will end up having all in one apps or voice wants like to a star trek replicator. Fun things are coming, but humanity will need to adjust hard - best case scenario.
Depends on where you draw the line for "near frontier model", but the current leading open-weights model, MiMo-v2.6, has exactly this problem. They released a fix on Hugging Face[1], but as of two days ago the model available via Xiaomi's own API still sticks in reasoning loops.
[1] https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Pro-MOPD
This seems to be such a Google thing. Anyone who has used Gemini for long enough has encountered these 'She loves me, She loves me not' spam loops
I don't want computer languages with complex syntax anywhere near me: that's why transpilers of those very languages to languages with a simpler syntax (usually plain and simple C or directly "human" assembly) are in my opinion so much important (we have a c++ to C transpiler and a microsoft rust to C transpiler, coded with the assistance of AI, and those should only improve from there).
For me, the current main issue of "AI" is their gatekeeping using the whatwg cartel web engines, yes, technical access to "AI" will have to be hardcore regulated if it becomes ubiquitous, and I said 'technical access', not AI itself, the difference is abysmal, it is an interop issue.
This is a clear step change, I'm genuinely amazed.
Its answers are to-the-point, logically structured, clearly written. Like a really well made mini essay on your question.
It also really likes to build little simulations / models that you can tweak the parameters on. Amazing to explore how different assumptions affect economic scenarios, for example.
It's also great at picking (sourced) illustrations.
I'll be using this for everything that isn't coding from now on.
As an example, I recently had an OCT scan of my retina because I'm a “glaucoma suspect.” I'm a glaucoma suspect because I have an unusually high cup-to-disc ratio, which is a pattern typically seen in glaucomatous patients.
As part of this OCT scan, you get a report back on your RNFL, which is basically a report on the health and thickness of your optic nerve(s) and retina(s). (Dramatically oversimplifying, of course.)
My ophthalmologist struggled to explain my OCT scan results to me and instead sort of shrugged and said, "It's probably because you're a high myope, but we'll keep you coming in once a year.”
But my scans look horrifically bright red — meaning I'm in less than the first percentile of retinal thickness for my age. Obviously concerning enough to do my own research…
ChatGPT 6 Pro and Opus 5.5 have struggled to make any sense of the clinical implications of these scans. In contrast, Argon was the first model to:
1. explain how the OCT machine works
2. detail the physiological mechanism that causes the cupping and thinner RNFL layers in my scenario
3. sum it up into a clinical recommendation (similar to what my ophthalmologist recommended, with the addition of basic supplementation and lifestyle modification changes)
4. provide an interactive visualization of the topography of my eye vs a glaucomatic eye: https://optic-nerve-3d.netlify.app/
I'm blown away.
It's kind of infuriating to hear Dutch people cite these chatbots without bothering to check https://www.thuisarts.nl/.
Sometimes I don't know a topic well enough to even know the right terms, or sources. But an LLM can help me become familiar with relevant concepts, and then I can more easily find the authoritative sources.
Sometimes I don't know a topic well enough to know how far it is from my understanding. An LLM helps me quickly learn how closely I have the relevant foundational knowledge: if it's absolutely gobbledygook, then I realise I've probably bitten off more than I can chew; if I half sounds familiar, then I can more confidently begin independent exploring of authoritative sources.
The problem with google'ing 2 years ago was a combination of SEO hijacking and sheer ignorance on the user's end.
Critical thinking is a necessary ingredient regardless of the tools being used
I'm aware, both of my parents are retired GPs.
Back when they retired, about a decade ago, they already had spent a decade complaining that the a constant increase in excessive administrative work got in the way of the actual work of being a doctor that helps their patients. So much that they had joined the strike (remember when that was a big deal because it happened for the first time in Dutch history?)
Anyway, the situation got more complex since then but a lot of pressure is also still administration. Yes, now we have assistants taking on more complex tasks leaving the "important" things to the real doctors, but I think that the need for that is also is in large part due to doctors not getting enough time to be doctors.
And I feel like automation will help fight back at that about as much as IBM's 1967 prediction came true that their office machines would fight back at the "paperwork explosion". Which is to say: Jevons paradox said no.
[0] https://www.youtube.com/watch?v=_IZw2CoYztk
[1] https://en.wikipedia.org/wiki/Jevons_paradox
But you're right that you can't be 100% certain. Isn't that true for human-to-human explanations too?
Part of it is using good judgment. Is the AI's claim reasonable if it came from another person? If some random person gave you a 2 page paper proving P=NP, would you even spend time to look at it, or would you simply toss it without caring to understand what error they made. Much like how scientists (and even the patent office) simply refuse to entertain any perpetual motion machines. Maybe if an expert in the field proposed it, and even that would depend upon how ("Hey, I found a source of free energy!" vs "Hey, I'm working through these equations and the energy term isn't balancing, can you help me find what energy source I'm missing?").
This does pose a problem. Those who are the best at understanding the limits of their knowledge and at teaching themselves need the AI's help the least but are in the position to be able to use AI with the best chance of not being misled. Those who lack these skills and would benefit the most from a personal educator, thus receving the biggest benefit in theory, are the ones most likely to fall victim to the errors made by the AI.
"Just see a physio" misses that:
* I want the physio's advice during workouts
* I want the physio's advice during moments of pain
* I want the physio to explain things in more detail, but my 30-60 minute appt is not enough, and it's difficult/expensive to book them for longer.
But, as a patient, it's hard to find comfort in that. I think that might be what drives the craziness in “non-traditional” approaches to treating disease. Could it be that people using fruititarian diets and essential oils for cancer are doing so because they don’t understand enough to bear a “wait and see” situation?
Argon explained the problem to me in such depth that I could be content with my basic treatment plan. ChatGPT went way off the deep end, suggesting things like high-dose niacinamide treatment (which is admittedly being evaluated in clinical trials, but comes with a host of other side effects).
Thanks for sharing and I hope your eyes remain fine.
FWIW, I haven't seen the Gemini 4 explainers, but Codex and (particularly) Claude have been pretty decent at this for a while.
Explanations give you a mental model: when I move this parameter this the effect it has. This just shows you the obvious, make a thing bigger, it gets bigger on screen. That's not an explanation.
Models are quite horrible at explaining things. They produce local narrow views that don't take into account the mental model of the listener nor do they attempt to produce the kind of long-term knowledge gain that a teacher does. I taught for over a decade.
This is why models are extremely good for people who already know what they're doing. That narrow why is all you need, like, tell me the 3 sentences that describe the mechanism behind this paper, and I'll get the rest. But building the scaffolding onto which to hang a lot of knowledge?
Give it a try. Try to have a model teach you something new. Not like "how do I tie a knot" but like a new area of math where you're going to dedicate 3 months of your free time to learning it. The results are pretty horrible.
Google have strict limits on new model this time and are pretty conservative. As we know they spent lots of time on internal testing since release.
All the signs mean Gemini 4 argon is a pretty large model than previous.
why? is it not so good on that? or is there something better performance, or better price for same performance?
That said, I'm extremely impressed with Argon thus far. I take back everything bad I've ever said about their AI strategy. Argon is very good and very fast. It might be competitive at coding, but it's too early to say.
Anecdotally, I have had Opus 5.5 decline a reverse engineering effort into a Zigbee gateway that I own.
DeepSeek did not have any issues with this using Claude Code as harness.
Found this especially impressive because backpack material related research has been something I know models can still get caught up on and the models being decent at web research in coding nowadays does absolutely not apply to these task equally Recommending nylon canvas shell suppliers or waxed cotton when PVC coated cotton is explicitly request, that sort of thing. While not perfect in reading coding docs, models are far better there.
Like with front end and code “parsability”, feel like you can very clearly point at which labs use what training set, with many often sharing the same suppliers. Once I noticed that, I became very bullish on specific training sets over emergence as the source for current gains.
Over the past two weeks, I have started using a locally modified version of deepseek-harness (modifications include disabling telemetry and querying a provider's model list dynamically through Openrouter, instead of relying on the pre-packaged pi-ai manifest that comes with the base code). So far, that has been the best experience - I really like the caching, tokens/sec stats that it provides. (EDIT: I don't know a lick of TypeScript but AI-driven modification of the harness is good enough for me to make these tweaks).
Sandboxing is a definite issue for me. As you perhaps know already, headless operation is often not an option in graphics/game programming which requires actual render-surface instantiations from windowing systems and then input-context handling (keyboard/mouse/gamepad) as well. So I run most development on a VM. But part of my test-suite and smoke-tests rely on runninng it directly on the host OS (which is not ideal). I'd love to hear suggestions to avoid this.
VFIO/GPU passthrough maybe?
You can apply it to the RC2 branch so:
git clone --branch dsh-v0.2.0-rc.2 https://github.com/deepseek-ai/deepseek-harness.git
cd deepseek-harness
git apply /path/to/dsh-v0.2.0-rc.2-to-local.patch
I'd recommend asking the harness to modify itself though (or a cloned repo in another directory) rather than applying the above.
As a side note, I am kind of sad that basically all models nowadays are RLd to hell and back specifically for coding and deep reasoning, and as a result, they lack in creative writing, human-readable prose, and even general knowledge and abilities (for example, qwen 3.8 27b is a beast for coding, but reportedly falls off the cliff in anything else). Gemini 4 Argon seems to be an example of what you get when you DON'T do that - Google seems to be one of the only labs optimizing for general performance.
I also fear that models cannot be excellent in everything at the same time. I fear that there will always be a heavy tradeoff between what specific field the labs optimize for VS everything else. I don't know.
(Disclaimer: I work at teh Goog and spend... a lot of time with these models :)
I mean you'd have to crunch the numbers, but Anthropic would need one helluva advantage in efficiency before that would be cheaper to google than the full stack in house
"I'm interested in understanding the rate of VOC dissipation from fresh oil paint on a surface versus the ventilation rate in the room in ACH. I would prefer solid academic studies of this, followed by analytical evaluation, followed by references to non-quantitative sources. Tell me what's known about this."
that I had run in ChatGPT earlier this week. (Yes, we just had our bedroom repainted, and it's TERRIBLE and I hate it, why do you ask? :-).
I didn't try it first in Gemini Flash, but I did try it in whatever model is used for Gemini Live and the answers were terrible. The ChatGPT answer (I think on medium) was pretty good but it threw in extraneous formulae that weren't actually germane to the analysis. The Argon 4 query I just ran was honestly the best of the answers I've gotten -- it was clear and really well-structured from the perspective of "here are the three big phases that researchers break down paint drying into and what the differing effects of ACH are on those phases".
I'm totally going to give this one a try, after basically giving up on 3.8 flash for most things despite having a free $20 pro subscription (from buying a phone).
Edited to add:
Argon4 transcript: https://share.gemini.google/18Gr5liH1YWR
ChatGPT transcript: https://chatgpt.com/share/6acedc72-ff18-83ea-8909-104cdbc6ac...
I've actually spent some effort trying to make them halfway decent placeholders by putting together a skill based on diataxis framework (https://diataxis.fr/) that at least got everything into the right location
just took a pass at rewriting them w/ argon and am very pleased, big improvement
Anthropic controls the full serving stack, so it can change precision, hardware, kernels and settings whenever it likes. Reducing the KV cache precision, for example, saves a lot of memory (and money). It may even be possible to quantise the model on-the-fly when loading it into memory. They don't necessarily have to upload a whole new model.
Even if they can't do the same on all platforms, we can't deterministically compare them anyway.
At any rate my point remains that this would require a conspiracy between 3 companies.
I've been trying it out via the API and the model has been a delight so far. Clear in the head, not RL-fried, distinct writing style, and big model smell. That said, the em-dash is back w/ this model.
That said, I would be so frustrated with Sundar Pichai if I was on the Google team. The botched rollout of this model seems to be entirely a matter of capacity, and Pichai has been more than happy to give it to anybody but his own team.
I haven't tried Argon, but for writing, I actually prefer 3.8 Flash over any GPT and Opus.
I know it's a matter of personal taste but it reminds me of a grad student that tries to sound smarter than he is by being dismissive and opaque.
Great model to architect and resolve eng. problems in a harness loop tho. Poor handoff.
I don't want ELI5, I just want the model to go straight to the point, like a colleague human expert would do.
Strange. I wonder why? Have you adjusted 'roles', added 'ask clarifying questions'? This is most peculiar.
> You gave me a long answer but it didn’t answer my question: can we safely remove $X
So obviously, government documents, top publications from liberal arts scholars and STEM > Wikipedia > Reddit (and within, r/AskHistorians > r/all, etc.). The first group is the one most likely to (over)use correct / fancy typography.
I have a related pet theory: the telltale sign of AI writing is being too good for the context it's found in, in terms of vocabulary and style as well as structure.
What is jarring about this is precisely the quality - we associate certain styles with important (or self-important) venues, so it's uncanny to suddenly see random shitposts, copywriting spam, or regular chat replies look like work of NYT bestselling authors or legal scholars - it feels wrong in places we expect basic language, grug-brain reasoning, and lots of typos.
Also, what makes the above much more noticeable, is that AI systems don't have cross-session stylistic memory. From the POV of an LLM, that perfectly fitting, masterfully crafted starter is the first and only time it used it. From the user's POV, it's the tenth time they're seeing it in the span of an hour.
LLM intentionally randomize output to cover up all the plagiarism, but this largely destroys the consistency that cause writers to feel unique. Larger patterns still emerge but it just feels stale.
LLM’s will happily regurgitate multiple paragraphs practically verbatim if you don’t specifically design them not to behave in this way.
Because ultimately when you design a system to best predict the next output based on a given input perfect replication of prior training is the correct response mathematically.
Models absolutely do have voices. Spend some time using them, and you'll see. I can easily tell Claude-written prose apart - a consequence of using it daily.
> not because of plagiarism concerns
I’ve second hand accounts of meetings focused on exactly the problem of how much verbatim text was being spat out and what to do about it. Some versions of specific models are far worse at this but it’s a huge legal liability not to address it.
Doesn't change my point anyway - even if post-training, labs naturally want the LLM to come off as smart (well, except for Grok). Post-training is where they "untrained" em-dashes and such, due to complaints of users.
Agent: _wall of text_
User: yeah sounds great, do it
Which has the predictable outcome of biasing towards wall of text responses.
Em dashes ended up as one of the first victims of llm age
In other words, since the very start of computer era, the use of em dashes was always a signal.
Every point is delivered with grandeur and rhetorical flourish. It is hard to reward 'negative space', a simple writing, A to B, vanilla pattern.
I don't ever recall seeing an LLM using them incorrectly, as opposed to simply at high frequency but correctly (although I'm sure some low param or early models must have done so at some point).
Way too many bullets -> way too many emojis -> way too many em dashes
They aren't a complete rarity by any means but still relatively uncommon compared to the typical punctuation one sees in routine usage.
I loved em dashes back in high school and my English teacher would have definitely returned an assignment with a lot of red marks if I used em dashes as frequently as an LLM (speaking from experience).
"Binds the downstream RF port template—which we configured earlier—to this RPD profile, applying all global DS settings and channel structures to the node's downstream hardware."
Do you see the problem in comparison? The LLM text is assuredly not how humans write. Well, maybe you would find similar examples in the classic literature, but definitely not in a common internet texting or simple articles and posts.
If you just want good llm text, it’s great.
I think there was a lot of misplaced hate of the em-dash due to all the low-qualify AI slop people were trying to pass off as their own for a while.
I'm not entirely sure you're correct. Your claim boils down to "Writing natural human prose" vs "Broad use of complex software and spatial awareness". My gut feeling is that there's more untapped and fruitful applications in the second group than the first - at least in the short term.
Given FelonyBench, do we also have a FantasyBench?
At the same time, OpenAI had a disastrous dev day that led a lot of users to jump ship to Claude.
Couple weeks later, Gemini-4-argon is still unavailable on anything less than the ultra tier, which means there's no natural funnel to get ppl to pay for the more expensive ultra plan other than "trust us the model is good".
If Google had argon broadly available earlier this month they would have picked up a ton of new users.
Models move so fast now and Google is very cautious about releasing them.
What sucks, hard, is googles ability to deploy direct to devs and customers at scale when the deployment is not a longstanding Google project.
Case in point: I’m an ultra subscriber and see in my app right now an ad to try Gemini 4; meanwhile the only models I have available are 3.1 pro,3.6 flash, and 3.8 thinking.
Clicking the try gemini 4 link takes me to a demo prompt that loads 3.6 flash. Srsly
I won’t even start telling you what getting api key based access together was like a year ago.
Please google, just make it easy for me to give you money and get what I paid for.
Unless Gemini CLI greatly improves and Argon works well in it, this probably won't make a dent in the coding market.
Ofc I know this isn't your fault in person, but I really wish Google would get its shit together.
...As a long time tech enthusiast I should probably stop being so delusional. Heh (cries internally)
1: lol I know who cheers for a trillion dollar company but I'd rather not cheer for OpenAI
As a Google employee my defense is it's hard for a company this big to move fast, but I can understand how Google customers would be annoyed.
The top level comment blamed cheerleading but let's apply critical thinking for a second as product people and look at the funnel:
1. Potential users sees new release 2. Potential user takes time to install and set up Google harness (you already lost 90% of people here, the funnel better close quick!) 3. Potential user cannot use new model.
Then what? You hope they developed good will towards your model through this experience?
You stuck a two week wait in your funnel? Is that the definition of not having your act together or what?
We're on HN, how many of us have heard from Google staff this kind of stuff is a result of them growing so big that AI Cloud has to compete with Vertex Cloud, and AI cloud exists because Vertex Cloud needed 2 years to get OAI/Ant style API keys working and were losing dev mindshare in the meantime?
Then basic mistakes (inferring wrong information from the given text, hallucinations...). Gemini is okay for some absolute basics and the rest of the work goes to actual frontier models.
Narrating this as if it’s the game of thrones is a fun exercise, but it’s just becoming commonplace now that models jump ahead of each other on a weekly to Monthly basis.
This was part of the issue, as stated in their comment. We couldn't jump around to try it for the reason that is wasn't made available to, as you say, "all the power users".
Do you really think people in google are happy about how long it took to release this model?
We’re all running on Astra medium the whole week and loving it.
I’m curious what I missed.
$500/month for 40+ hours a week of Astra usage, with ultrafast mode available when you’re near the weekly reset has made our jobs so much easier. Our product has some sciencey bits as well, ands Astra kills it as a rubber duck in this area.
As a consumer whose 20x account got downgraded, I am a bit sad. I can’t justify $500 a month on personal AI spend. As an engineering manager, I could not be happier with OpenAI’s services. Also, Codex App Server is amazing.
In my experience, this has always been the advantage of OpenAIs models.
Throughout iterations, Claude is better at pure coding, but the GPT models are better at "scientific reasoning".
Hey, account created 39 days ago. Did you really have to tag that on after all the other stuff? Could you make it any more obvious?
That said this thread is hilarious because it's just an open ai sales manual debating with an anthropic sales manual. It's funny, but it's not, because the truth is that you people are just shitting all over the commons. Someone needs to kick you out but that's not going to happen here.
Claude refuses to help you build your own harness off a subscription, and OpenAI actively encourages it with Codex App Server. So you’re just in an AI IDE customized to your job and specific needs ALL THE TIME. You know that good feeling you got when your CI pipeline was finally working, back when that involved speed reading documentation and guessing, or staying at work until 10PM? You just get that all the time because life is constantly improving. And your tools are beautiful and riddled with Easter eggs you put in them. There’s so much more time to test and talk to stakeholders because you are able to distribute the thing you are actually good at -solving people’s difficult problems- while following best practices. You used to have to weigh those two every day and it was terrible. And everything was always late. You had to cut so many corners you were delivering circles.
Now life is fun and I’m over-delivering. Constantly. With super high code quality, better test coverage, easier audit prep, and I just explain what I want (technically) at a high level to AI. I still have to code review, and it sucks. Not because the AI needs it, but because auditors need it. I think I can make it faster and easier in the near future with another custom tool.
I actually have time to make architecture diagrams. I have never had enough time to build that level of documentation.
With Fable, I felt the promise of this coming. With Astra, I feel like we finally had the resources to fully professionalize and supply our software team.
We actually need to hire more people next quarter.
I am with the creator of Rails on this. It is a time of joy. Now is the time to become more ambitious.
Oh yeah. Completely forgot about that. Big headlines and big claims, but no one could actually use it and report on it.
Does seem like they don't quite grasp how the hype train operates. Or maybe they don't care as much.
I mean, I heard rumors too that it would be ultra only, but that doesn't seem to be the case, unless you are specifically talking about antigravity?
Realistically, market adoption isn't going to be the result of terminally online people (like me, I confess) following things way too closely on social.
edit: just a few hours after this comment I got access to Gemini 4 on the Gemini app. Yayy! :)
https://blog.google/innovation-and-ai/models-and-research/ge...
Can you elaborate on this?
Reddit thread with 247 comments here: https://www.reddit.com/r/codex/comments/1wtgv3z/dev_day_l/
Aside from that I'm not sure what was "disastrous" - maybe the lack of a new model release to compete with the just-released Claude Opus 5.5?
'backlash' is underwhelming. They single handedly murdered 5.3 codex which was great by making it slooow and duuumb, and pushed people over to their new ones. 5.6 was a decent candidate but it was so slooooow.
And then double or triple the cost for Plus and Pro by nerfing their effective power? I never looked for an alternative faster. They really screwed the pooch.
Been quite happy with deepseek v4.1 flash, muse 1.3 and haiku 5.5.
- Dots. People don't understand it. Their presentation didn't explain it properly at all. Additionally, many people thought it's stupid, and that the cutesy mascots are dystopian.
- Voice mode didn't work in their presentation. The demo failed.
- 200$ plan goes 20x -> 10x usage. 500$ plan introduced, 25x usage. The contrast is stark.
- "Ultrafast": 8x pricing on subscription plan (6x API). That's 80$ input, 400$ output, 100$ cache write. Hello?! Additionally, since it generates 8x faster, it consumes your quota 8x faster. Per wall-clock minute, it's 64x more usage spend per minute of generating. People report that on the 500$ plan, their weekly usage is gone in 40-100 minutes.
- People believe gpt-6.1-sol was planned to be released as astra-minor, but OpenAI renamed it at the last possible time, because gpt-6-sol is a disaster (this I can get behind)
The results are a bit disappointing given Googles scale and training data.
The results are on par with cheaper Chinese open source models to be frank. https://cliphou.se/benchmark/
noooooooo
- Natively multi-modality
- Enormous context windows
- Reliably cheap prices
It's like they don't release a model until it passes the above checklist.
Most providers have some mixture of these three occasionally, but with Google, every model is consistent. This kind of consistency is really nice to have.
Outside of that, behavior is quite good, a bit slow, and definitely different than both GPTs and Claudes -- which is appreciated.
do you have to pay those cheap prices eventually or can you just accrue an unlimited amount of overdue charges?
https://arena.ai/leaderboard/agent/chat/pareto
For almost same intelligence (+15.56%, +15.18%), drastic difference in cost/task ($3.51 vs $0.41)
From the graph it's a flat line, but looking at the point yields a negligible increase with no error bars (indeed if you look at fable 5.1 it's measured as lower than opus and argon). This just seems like bad data or blatant favoritism at worse.
For chat, yes. But switch to code on the same page, and the picture changes. OpenAI takes the lead.
Argon build: https://html.non.io/lcars-argon/
GPT 6.1 sol: https://html.non.io/lcars-gpt-6.1-sol/
Opus 5.5 (currently SOTA for this): https://html.non.io/lcars-opus-5.5/
Original designs: https://diffui.ai/app/canvas/5093e689-1e74-4f26-b632-2a4500f...
(Chrome, also incognito)
As soon as gpu acceleration is disabled, the webpage goes from 60fps across the three systems I tried to ~7-20fps when software rendering is forced.
I suspect if you go to chrome://gpu you'll find something different from what I have here: https://image.non.io/a47a1b03-1b87-4fd5-92a2-b39e62b1cfd9.we...
So that all being said, a normal website should work totally fine on low power mode, and this should definitely be a negative on the Opus implementation.
I would disagree with this based on two things:
1. the prompt, which explicitly states to render the planets in the background in 3js (thus the gpu implementation).
2. the purpose of this, which is a test of its ability to convert an image design into a html/js/css implementation that matches it as closely as possible. I've talked about this before when it's come up[†]. The purpose is to be a good test, not a good design or website.
For reference, here's the full prompt used:
> Create a web page implementation from the following instructions: https://diffui.ai/build/Futuristic_Interior_UI_build.md?auth... dont reference any other folders that have similar builds. Focus on the flowing SVGs. Animate elements that make sense to be animated. For the starships that have planets in the background, ask for high quality images with the windows as a transparent background, and recreate the planets behind them in 3js. It should be one single page application that I can just open the index.html file to view.
[†] https://news.ycombinator.com/item?id=49997135
[‡] here's a copy of this since the diffui build instructions wont be accessible without an account: https://non.io/lcars-build-instructions
FWIW, I have literally the exact same configuration (M3 air, google chrome). Here's a recording on mine where I'm seeing 60fps: https://non.io/video/video-3587018401.mp4
Edit: The argon built site pins 1.3 cpu cores. The Sol 6.1 site uses 0.18 cores.
Here's a vid of both opus and argon side by side, which might be more viewable on mobile: https://x.com/pwnies/status/2110058987462988152
IE for Argon, the third section doesn't have a top view of the ship with labels going to each section like what's in the design. Opus has it, and animated it as well. One failed to do as instructed, the other went an extra mile.
https://dach.peerbench.ai/compare?models=google%2Fgemini-4-a...
But id the progress of decoding it is legit, you should write something properly about it!
What kind of progress was made with Argon with the Voynich?
It looks like you've managed to decode it? Is that legit?
Eyes glazed over again.
Did you even read your own site before you posted it?
One of the first pages I opened:
> Smoking-gun proof of plant-label ingestion: in Row 2, the central cut root is labeled okol (f88r.15), and directly underneath it on Line 2 (f88r.19) the recipe writes...chokol daiin qoekol qoekol qockhol okol cheol...!
It's just gibberish.
It only makes sense in a wider Claude session, and even then it's like he's got Claude set to super-boot licker mode.
https://www.bridgebench.ai/nerf-bench
https://marginlab.ai/trackers/claude-code/ https://marginlab.ai/trackers/codex/
https://github.com/ninjahawk/livenerf
https://isitnerfed.org/
Since then they cut the price but the intelligence (of 6 and 6.1) has gone down to Sonnet 4 levels. I am back in a world where the model is lazy, cuts corners, ignores existing coding standards and context, makes stupid mistakes etc.
I am not sure how those models still rank so high in benchmarks when real-life usage is so obviously bad.
State of affairs [1]:
> My whole project and career depended on the 3.1 pro workflow. Please, bring it back
[1] https://discuss.ai.google.dev/t/please-bring-gemini-3-1-pro-...
It thinks a ton, ~24k output tokens a run vs ~3k for Sonnet. It missed a solution by thinking its way into a 10 min timeoout.
Something to look out for - if you're using pi, it doesn't have Argon in its catalog yet and reports cost at Kimi's rates, about a third of what OpenRouter actually bills.
This is crazy if it can work accurately.
Here's low, medium, high. Even low is better than pelicans from any other lab: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
low cost 10.185 cents, medium was 11.098 cents, high was 17.166 cents.
The costs are similar because it used more output tokens than reasoning tokens for all three pelicans, even for high.
> If it's on HN and is a meme at this point, it will end up in the training set.
> It's kind of fun to imagine that there is an intern in every AI company furiously trying to get nice looking svg pelicans on bicycles.
Google listened and assigned a few interns, I guess, for the task
Figure of speech.
>Why not just generate svgs with any editing software?
How do you do that from the whiteboard/notebook?
Everyone says that until ms, apple, google, etc introduces this polished feature as part of their tablet imaging editing/document collaboration office suite. Then everyone can;t get enough of it.
> It's a small task whose result is an input to something else - ideal for a tool for LLM (or script)
Have you actually tried this? If it's a small task then why can't LLMs actually do it? Kind of like the pelican test. Every LLM can do a SVG pelican but most of the results are unusable in practice for production/productivity purposes.
Every LLm can extract some half-broken SVG hallucination of your jots, but I'm asking for an app that's polished and works accurately 99% of the time, which means a dedicated app/model tuned specifically for this task, and nobody makes this yet.
I don't feel like I'm a representative sample of general userbase, so I won't dispute this too hard. For me, it's rare but it does happen. Still, 90% of my mobile experience is "how is this so broken, why is it missing the obvious follow-up feature that completes a workflow, and why, again, am I not using a laptop for this?".
> If it's a small task then why can't LLMs actually do it?
What if they actually can?
This is exactly what Karpathy was talking about the other day[0]. If your only LLM experience is the default chat app in chat mode, then this doesn't make sense. If your experience includes using agentic harness like Claude Code/Cowork / Codex/ChatGPT Work, then it's obvious.
The LLM app in "chat mode" will just zero-shot you a pelican, or digitize your notes into SVG, and maybe improve on it a bit if you complain about the result. The same app[1] in "agentic mode" will digitize your notes, refine the SVG against the original image, reduce it to probable strokes based on high-frequency patterns in the photo and knowledge of how people write, OCR it, save both the text and SVG to your library, then extract parts you wanted and insert them into a document, typeset the rest of it, and print it on your printer - all in response to a single prompt.
LLMs in agentic mode are very good at stringing long and advanced workflows; today, if you give them shell access or computer use, they will use regular apps for you if that's the easiest way to accomplish the goal.
> Have you actually tried this?
Yes![2].
That's why I can confidently say: there is no limit to this pattern. LLMs can handle apps just fine; giving them tools just saves time and tokens.
Another example: I'm currently resurrecting a 30 year old game, and the other day I asked Claude to hack on a way to run it headlessly, so I can use my computer while it plays with it. It did, and then immediately leveraged it to just causally run several instances of the game in parallel, entirely in the background, whenever it needed to playtest something.
--
[0] - https://news.ycombinator.com/item?id=50047422 - was on the frontpage two days ago.
[1] - Claude and ChatGPT now have both modes in their mobile, web and desktop apps. I personally use Claude Code a lot, but exclusively through these surfaces (I can't be arsed to deal with TUI at home anymore). IMO, these tools are already within reach of regular users - the users just didn't notice it yet.
[2] - Not with SVGs, but equivalent with voice recordings, and yes, Claude Code set up and is managing the whole local processing pipeline. Recordings go in, Markdown (or more advanced signal processing) comes out, and in itself, it's not a task, but merely a step.
One fictional example of what I mean: lets say I draw someone holding a purse. That would be a shape of their torso, with the purse overlaid. I could move those independently, and moving the purse would show the torso underneath.
However, using a png to svg, it would often just outline everything. So it's not a torso shape with a purse in front. It's a purse with a torso cut out around it, maybe even merged together as lots of individual shapes. If I were to delete the shapes making up the purse, it could be quite random what was left. Maybe some of the shapes did a double duty. Maybe it's a thousand small squares to make the purse. Maybe there is just a hole behind the purse etc.
So in the latter case, the svg isn't made up of logical elements, it's just made up of shape after shape after shape to mimic the pixels. But it's not usable as vector.
> How many pelican riding bicycle SVGs were there before this test existed? What if the training data is being polluted with all these wonky results...
But Simon has demonstrated before that asking for any animal doing something silly will still get similar results from the same model.
> That's the result of training on the same datasets and scraped web contaminated with other models' outputs
Maybe I'm misinterpreting though, or just semantics. I would consider contamination, or whatever the first example was that they all matched, as specific training data, along with all the duplications of. I doubt it's model collapse to the same pelican, without the same original example.
Expect Google models to be benchmaxxed as much as possible, but don't expect commensurate performance on real-world tasks.
This is not true. It is needless negative hyperbole that IMHO doesn't belong on HN.
If what you say is true, then the next set of benchmarks will be written to illuminate the true reflection of real-world tasks. Then those get benchmaxxed so managers can show "numbers go up."
Repeat enough times, and you get materially and genuinely improved products.
The burden is on the press, outside researchers, and general tech industry to both write and care about the metrics that matter.
In large companies (Fortune 1000 etc.), you don't have much of a choice, the culture was eroded long ago, and corporate governance by metrics is a lesser of evils. But thankfully, not all companies are there yet.
They look remarkably good, sure -- but not perfect.
This is the sheet: https://fernandoipar.com/content/music/poly-d/gemini-argon/c... (the annotations on it are by me)
And this is what Argon created: https://fernandoipar.com/content/music/poly-d/gemini-argon/b...
For this simple prompt: https://fernandoipar.com/content/music/poly-d/gemini-argon/p...
I'm honestly impressed.
Subtle highlights on the metal frame, sweat beads, jaunty little hat…it has style. At least compared to the rather literal ChatGPT outputs. https://static.simonwillison.net/static/2026/gpt-pelicans-gr...
Is the prompt context telling the models what their effort level is, and the ambient presence of that concept of ‘high effort’ in the context window is leaking through and affecting the portrayal?
https://huggingface.co/Qwen/Qwen3.8-27B/blob/main/chat_templ...
xhigh: "Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer."
low: "Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration."
I note that even these bikes are not actually rideable. For a bike to be stable, the front wheel axle needs to extend forward a little ahead of the fork - the fork must ”bend forwards”. This is what actually keeps the bike upright, not gyroscopic effects or some such. Given LLMs are generally good at basic maths, im surprised they get this wrong.
Maybe a bit less so in this case though. As per the announcements, the model was actually trained for this prompt, so was fed some training data. I can well imagine the data there was just not so good.
You can also ride a negative rake fork. They are less stable, yes, but a far cry from not actually rideable, cf https://www.instagram.com/yourforkisonbackwards/
People drawing bikes, scroll down to see they having made renders of all the drawings. Hilarious
I'll also note that, using a piece of paper as a straight edge against my screen, there is a very slight forward curve on the low and a bit more on the high responses. Medium does appear to be perfectly straight.
But even with zero offset, there is another thing in play. The steering axis is at an angle, a fork never points straight down. So even with zero offset, you still get a trail, which is the difference between the contact point of the wheel at the ground, and where the steering axis meets the ground. This trail is really what matters in the end, not how you get there.
See https://en.wikipedia.org/wiki/Bicycle_and_motorcycle_geometr...
But even better, play with (ctrl+f) "trail" in this simulation: https://ciechanow.ski/bicycle/
There you see the point. Even with zero offset, there is still a trail.
You're nit-picking on a detail no one actually cares about.
https://www.walmart.com/ip/Huffy-26-inch-Rock-Creek-Men-s-Mo...
(apologies if I'm misunderstanding).
Whether having zero offset or not isn't what matter in the end, but the trail. And you can achieve that with little offset as long as the fork is angled.
(so the grandparent is a bit right in their critique, but not for the reason stated)
1. Modeling and analyzing spinning, three-dimensional objects goes far beyond "basic maths".
2. The prompt was for an SVG of a realistically impossible situation (pelican riding a bicycle), which together already implies a stylized depiction. Why on Earth would you expect it to generate a 3D model of a bike, do a physics simulation on it, iterate until it satisfies certain contraints which are only hinted at in the prompt, and then project it down to a simplified 2D cartoon drawing? That's not how LLMs work, and that's not how we want them to work.
https://en.wikipedia.org/wiki/Pelican#/media/File:Pelikan_Wa...
Argon: https://jonclegg.github.io/pacman-bakeoff/dev/#gemini-4-argo...
Opus 5.5 https://jonclegg.github.io/pacman-bakeoff/dev/#claude-opus-5...
Also, that first tweet contains a commentary that, for me, explain a lot about the public misconception about how current AI works:
> wait, does this mean you guys are actively training svg generation? i was mostly amazed by it thinking the models doing better at it by itself due to becoming better at reasoning.
No, it's not better reasoning that drives current AI, it's better examples we have been giving them. Whole army of people generating content for the frontier labs.
Also, reasoning improving the out of distribution part, like computer use, tool use etc.
Another case where allowing oneself to anthropomorphise LLMs a little yields better intuitions.
People don't get more skilled by raw reasoning, either. They improve their skills with better textbooks (pre-training data), more practice exercises (RL data), and generally goofing around (self-play).
It's a simple manner of prompt...generate concept art with nanobanana, measure, llm creates svg with text, measure the distance between rasterized svg embedding to nanobanana art....do post-training RL on a 3 million such images and matching embeddings.
I swear this will be a canonical example of Goodhardt's Law in action.
You have to switch to a new benchmark that isn't so easy to run a verifiable rewards pipeline on...Perhaps a longer horizon task where it's easy to mess up midway.
Yes, Simon has made pelicans a benchmark and now models have grown very good at rendering them. Is there a reason to believe that this isn’t manifesting as useful skill in general in SVG generation?
It would perhaps be slightly interesting if the prompt was run 100 times for each model. I doubt the pelicans would look identical.
Here's 'Generate an SVG of an armadillo in fishnet tights jaywalking on Mars' for Gemini 4 Argon: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
Some other models for comparison: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
1. the pedestrian traffic light's hand and "wait, citizen" text are not inside the light itself
2. The armadillo's nose is outside of the astronaut helmet
Isn't that just proof that the pelican has been benchmaxxed?
I've noticed that chatgpt is working harder. But still I prefer Claude, it just feels nicer to me.
However, how the front fork rake affect stability is I believe a little complex. I doubt this old article is the last word on the subject, but it's a great read nonetheless, and note especially the ultra-stability of the reversed fork bike!
https://physicstoday.aip.org/features/from-the-archives-the-...
As one point, the high pelican's arm is see-through/messed up.
They do look worlds better than previously, to be fair.
wasn't the whole point of pelicans is that labs *wouldn't* train for that?
so now the benchmark is ruined
Modern models can make nearly photorealistic games in threejs. So why do they struggle with these pelicans?
The test is always going to be hard though, because it's an absurd image to ask for.
UPDATE: I like this theory about using an agentic harness that can render and iterate better: https://news.ycombinator.com/item?id=50073879#50074395
this is a really profound insight
However, result quality really drops off once you get out of the top 5 or so.
(NSFW note: all buttocks are covered by fairly conservative bikini bottoms and show no genitals, bulges, etc.)
The pretty 3d stuff you see are agents controlling a Blender MCP in a loop where they look at the scene, think, adjust the scene, repeat.
(As I said in a separate comment above, results are all bikini-clad and not revealing. However, whether it is NSFW is up to you/your work policies.)
https://www.gianlucagimini.it/portfolio-item/velocipedia/
HN discussion: https://news.ycombinator.com/item?id=17699017
https://www.wired.com/2016/04/can-draw-bikes-memory-definite...
Are you sure they're not fetching free asset packs for that?
I even built an environment that allows execution of Python code to render with Blender but allows no other interactions (including networking, I sandbox that off). They can create good 3D models just fine - here's a Faberge Egg by Gemini 4 Argon: https://tools.simonwillison.net/blender-viewer?url=https%3A%...
Thanks for doing all this research!
Give me reverse engineering bench and agentic coding.
Leave making visuals to humans who actually have aesthetic sense unlike whatever made this mess
<https://tools.simonwillison.net/svg-render#%3Csvg%20font-fam...> <https://tools.simonwillison.net/svg-render#%3Csvg%20font-fam...>
- I was having trouble embedding the SVGs as links, and ChatGPT recommended Simon Willison's SVG Renderer as a workaround.
Just my 2 cents on the pelicans benchmark: I think it might help if you were a bit more explicit about not taking this one benchmark too seriously. I know you say that at times, but at other times (such as the comment I'm replying to) you elide it which can read as a seriousness about the benchmark.
Medium and high are fixies apparently, (or they have a worm gear, although the hub doesn't look big enough for that), since neither has a derailleur. That would also explain the lack of brakes at least.
Low is still obviously failing the "test". It's pedals are not actually attached to the bike either.
Medium has a handlebar that goes through the pelican's beak. So I'd say also failing.
For the high, the only nitpick would be that the bell isn't quite on the handlebars and the stripes on his scarf don't line up correctly.
I’ve tried the various alternatives (opencode, pi.dev) but for “have several physical machines, each running an orchestrator and 5-10 agents, and each agent spawning its own subagents to do work in parallel, all of that on a tanky GitHub project boards” level of work only Claude was doing work dependably.
I can have a few orchestrator leads which I talk to and brainstorm, and let them file tickets, organize them and assemble teams of agents to work of the various problems. And all of that with just a smidge of custom tooling on top of Claude Code’s default.
Plus don’t have a budget to spin all of this on cloud runners so I made sure to do as much as I can with local work on several machines I own. Was quite easy to achieve with Claude.
Do you have something similar built on top of a open source harness, or is it from scratch?
So that I'm not just relying on one or a few agents to accomplish a task. I work on the whole product and just drive the direction of tickets and their "what done looks like" requirements, and let the machine organize things on its own, usually 10-20 independent agents (each spawning their own subagents). I think i might have capacity to handle more, just don't have the subscription tokens to fuel it and the budget for the hardware to run it.
Haven't worked in BigCo so no idea how that compares to their tooling. I'm sure it works much more efficiently and on a bigger scale, just being curious about how it all plays out.
claude: Help me, Obi-Wan Kenobi. You're my only hope.
me: who is obi-wan kenobi
claude: oh you have to log in again.
me: <logs in>
me: play the whole message
claude: what message?
But ultimately I find it a silver lining more than a problem. I am building and changing tooling so fast, and the models tooling itself is changing so fast, that by the time I establish a workflow, I either A) have solved the original problem so no longer needed, B) have built other tooling that distorts the picture, or C) the labs have released something new and cool that changes the equation drastically so your previous setup is gone anyway.
For example I built some procedure on how to split work between fable and opus and sonnet to optimise token usage, then opus 5.5 came along and made all of this obsolete.
Not per “token usage” mind you, opus is still expensive, but more “effort per task time finish” kind of metric. If I want to close things faster while still being relatively within budget, it is just easier to send all to opus - it will now just solve thigns quicker so you still use similar amount of tokens / money, but quicker, which is a trade I’ll make any day.
And every time “the system is working” I either just stumble onto the next resource constraint, or I feel uncomfortable and start refactoring to make things better and inevitably destabilise the whole “machine”
We won't know the answer to this until IPO and until each company reveals the economics of their models (training and serving).
Google's years of research project into Indic scripts, image trainings, translation efforts & transliteration efforts is literally paying off. The AI adoption from non-English, non-Chinese population will only if you have these contextual localisations, which I think Gemini is poised to do well.
For example, you can ask for a shopping list for a marriage function in a town from Tamil nadu (the marriage custom varies by districts), and Gemini provides contextual and helpful answer (like what is the custom followed there, what is gifted traditionally, etc), whereas if you ask Claude/ChatGPT, it talks about marriage registeries or pretty generalised answers which is not relevant or useful.
Pretty misleading.
Okay, so I'll get Pro to try it.
I subscribe.
No gemini 4.
The most generous reading I have is that it's a mistake. But I also downloaded the PC app, and now I really wonder why I did that, it seems to offer nothing beyond the web experience and also doesn't even know its a local app. (When asked about its capabilities it always responded with reasons pertaining to it being a web app.)
However, my experience with the gemini, jules and antigravity apps was awful. So it is difficult to pin that on the model.
I wanted it to give me a proper comparisson for VPSes or server options and it took so much babysitting and it couldn't produce reliable pricing information or visulalisations.
I'm not going to 5.5x my subscription cost without being able to try it out first. To be fair though, I'm not going to 5.5x my subscription cost anyway, as my current usage would not warrant that! But that doesn't change the validity of my prior point.
EDIT: Okay, the AI plans page claims under pro - "Expanded access to Gemini 4 Argon in Google Antigravity." So maybe it is just a matter of it rolling out. Though Google has a long history of serious confusing and misinformation on those tier descriptions.
EDIT2: And they changed it again, and now it is explicitly excluded from antigravity for Pro users.
Even after using the free model in Google search talking to Anthropic or OpenAI models feels like talking to an ox.
As of my knowledge cutoff, three of the top state-of-the-art large language models are Google's Gemini 2.5 Pro, OpenAI's o3 (and GPT-4.5), and Anthropic's Claude 3.7 Sonnet.
And at least Flash 3.7/3.8 were both difficult to persuade that they have wrong knowledge cutoff.
I reported it as a bug to Google, but as all of my reporting to Google, no response ever. This is just stupid!
I'm not quite sure what their strategy is, because it looks like the models released just have tweaks on the same original training data, but nothing more. Basically, they decide not to release them and it doesn't seem to be a much different when they finally to decide to release.
- Try in Gemini
- Try in Google AI Studio
- Try in Google Antigravity
Will non users of Google products even know what these three things even are?
I have AI Pro, but AI Studio tells me I can "unlock" Argon by upgrading to Ultra.
Classic Google.
Is there anybody in Google having a look at how bad and complicated their payment system is? In all the other AI providers you just sign in, pay, get a key, done. In other clouds like AWS, you add a card, pay little to verify the card the first time, monthly bill, done. In Google it looks like you have to fight them every time for the very thing they should make as easy as possible: opening your wallet.
I would also advise against buying Gemini if you have any other Google services, because you get banned across board if they don't like something. To use Google it's best to first migrate off of Google.
Otherwise great product, I really like and would recommend Gemini. The usage is also very generous, at least Flash models.
I recently had to verify my YouTube account with a phone number recently because it requires this in order to set a thumbnail.
Questions right off the bat:
1. Why do I need to verify to post a thumbnail that will take 1MB on their servers but not to post a video that will take 100MB on their servers and be public??
2. Why do I need to verify my phone number on YouTube when it's linked to my Google account which has a verified phone number and 2FA etc.?
In any case, it takes you to a CSS-less page that asks you to choose between getting an SMS or being called to get a code. I choose SMS and it doesn't work. It shows an error message with no description just "something went wrong". I try a few times and it keeps not working.
I then switch to the call, and guess what! I have tried "too many times" and will have to try again later. Call worked the next day, but geez, what a mess.
For the lazy:
https://www.osnews.com/story/19921/Full-Text-an-Epic-Bill-Ga...
What’s so complicated about going to gemini.google.com ?
> Why isn't it called Gemini Desktop
There is a Gemini desktop app. Antigravity is for agentic stuff, similar to Codex.
i live in French Polynesia, a French territory, i'm a Google AI subscriber and i can't use Google AI Studio or antigravity cli because i'm in a geo restricted area ...
It seems Google has no issue laying out cables across the Pacific Ocean from/to Tahiti, they don't have an issue selling their services and plans there but i can't use AI Studio or antigravity cli because of geo restrictions ...
"Unavailable · Gemini 4 Argon is currently available to Ultra plan subscribers."
They now ask for your mobile number during signup. And it wouldn't let me register any more accounts with my number. So frustrating.
- Then I realized Gemini 3 was active.
- I tried switching models.
- I was presented with a $99/mo subscription offering.
- I scrolled down and clicked to get cheaper options.
- I chose the $19.99 "AI Pro" monthly option and paid.
- In chat I was able to use Argon. Cool. Seems pretty smart.
- I asked Argon if I could use it through opencode with my Google account.
- Argon said yes, but then told me I'd have to use the Gemini CLI as middleman.
- I told it I didn't know there was a CLI for Gemini; I'd just use that.
- It told me how to install gemini-cli.
- I authenticate via Google OAuth inside gemini-cli.
- Gemini-cli rejects my login, says the app is deprecated for subscription users, who must use agy.
- I install agy.
- In agy, I open the model selection menu.
- "Gemini 4 Argon is currently available to Ultra plan subscribers." Odd... Gemini 4 Argon is the model (included in my Pro plan) that told me to do this.
- I discover that Claude Opus 5.5 is included in the subscription. I switch to it and give it a debugging task.
- Within about 5 minutes: "Individual quota reached. Please upgrade your subscription to increase your limits."
- This is dumb. Google stole twenty bucks from me; I want my money back.
- I go to the Gemini web app to cancel my subscription. I paid on my phone, so I have to cancel on my phone.
- The Gemini iOS app has a "manage my subscription" page. I go there to manage my subscription.
- This "manage my subscription" page includes no way of cancelling.
- After a minute of confusion I go to the App Store > Subscriptions > Gemini > Cancel.
- My subscription will end at the end of the month (no refund).
- Then I have to find the complaint form so I can request a refund.
- But I did request one.
Anyway, "frontier intelligence" just wasted about an hour of my life.
for me, a google model can be as good as they want, in the end, they are still google, probably one of the most bureaucracy riddled big companys out there. evrything i see from then gives me feeling that evrything has been decided by a committee that do care more about politics and not customer needs. the shit one has to go trough to get the gemini-cli running is a good example of that.
That said, asking Gemini.google.com to walk you through the steps seems to suffice. In fact, one of AIs big perks for me is walking my through kafkaesque systems like Google’s to cut the shit and get to the result
but they don't want to share their compute with us. giving those resources to their ad targeting algos is more profitable than renting them to us. so they make it as hard as possible. they tried as hard as they could not to release a new pro model at all, but now that fight has been lost they at least need to make sure we don't use it.
What you did isn’t wrong, but is unfortunately very similar to what real scammers do. The heuristics behind these anti-abuse efforts are always going to have false positives.
Truly crazy how much Gemini still hallucinates, especially things which its own Google documentation or just a basic web search away. I feel it’s mostly solved problem with chatgpt/claude , but for me, with Gemini it’s almost guaranteed it will just make stuff up
They made using it so hard/unclear that myself and every other dev I knew gave up.
It's clear that they still haven't done any organizational self-reflection.
They had gemini cli, what happened to that? Was fine
Ended up with Soniox instead, took minutes to get started.
Very fast.
Makes mistakes. Corrects the mistakes quickly with direction.
Google Argon 4: https://share.gemini.google/T9r6tr2dDaTE
I didn't have time to make it into a full blog post yet. Here is a previous one.
https://generative-ai.review/2026/07/kimi-k3-rush-test-vs-cl...
Astra 6 is significantly better than Google Argon 4 and a lot better than Claude Fable
https://generative-ai.review/2026/10/catchup-openai-astra-6-...
(edited thanks to tekacs)
> Upcoming change to OpenRouter’s ZDR designation for Google authored models
> We’re writing to let you know about an upcoming change to how OpenRouter designates Google authored models for Zero Data Retention (ZDR) routing.
> Effective October 18, 2026, OpenRouter will no longer designate Google authored models as ZDR-eligible. After that date, requests that require ZDR will not route to Google authored models.
> Google’s standard automated abuse-prevention policies allow it to use safety classifiers to identify requests that may violate its policies and to securely retain prompts flagged for further review. Because this process does not meet OpenRouter’s requirements for a ZDR designation, we are updating our model classifications and routing controls accordingly.
> What this means for you If your requests require ZDR We recommend migrating affected traffic to another ZDR-eligible model before October 18, 2026 to avoid interruption.
> If you do not require ZDR No action is required for your Google model requests.
> Our current routing behavior will remain unchanged during the 30-day transition period ending October 18, 2026. > Read the ZDR documentation
> If you have questions or would like help identifying an alternative model, please contact us at support@openrouter.ai.
> Thanks for using OpenRouter!
> The OpenRouter Team
is it just 'Hire the right people'?
It's extremely inconsistent internally as well.
Look up the slime mold analogy for Google.
Google did a very good job on things that required longer research times and compute scale, note that Transformers, BERT & Scaling Laws came out of Google (though the latter were also worked on at OpenAI).
Conversely, Google is horrible at product, and without OpenAI, it probably would have taken many years longer to take off. I know this because Google had demos of this internally, but absolutely no stomach to release it, and they didn't believe in the API use case at all.
Which is to say, I think if Google was smaller, we probably would have gotten more SaaS, not more breakthrough research.
It's also worth noting, Gemini 3.5 Pro never came out.
Having worked there (but only in a small part of it that eventually got absorbed into Gemini), it's some combination of:
* The eye of Sauron (management) can make specific spots (e.g. Gemini) push harder (and attract the motivated people at the company and be selective)
* They hire a lot better than average (Generally from the top 10%, hotter places in Google are more selective)
* They have a promo process that gives career motivated people a way to prove themselves (not always aligned with things being on the right track ofc, and not always non-political, but they definitely do something)
* Redundancy in teams causes them to compete (not necessarily productively)
Of course, there are other downsides not listed here (insane bureaucracy for launches, GDPR/BCID compliance, etc, etc), but a lot of what makes Google possible is the insane amounts of cash Search throws off.
Perhaps one of their more successful chat clients, but that's all I can think of
The really annoying ones are the ones they buy, and then shutter because they can't make enough money on them fast enough.
I thought the nightlight that goes on when you walked under it was really neat...
What if the answer is "they don't"?
They don't.
If they did, neither OpenAI nor Anthropic would even exist, as the core of what they're all doing was invented by Google engineers and researchers.
Couldn't you just say what the price will be after the introductory period and when it'll take effect? Do they have some SKILL.md for these blog posts that requires the most important information to always be buried ten levels deep and hidden behind a login page?
$2/million input and $10/million output today. $4/$20 after December 1st.
It's perhaps a face saving thing to pretend you'll charge the same. Let's see if it actually happens.
Is Elon Musk doing their pricing?
Although given it's Musk, others may disagree.
It is price fixing, nothing else
I think at this point comparing raw token pricing is often apples to oranges.
When that isn't available, and the task involves "understanding the problem", "deciding what I'm actually after" etc. iterations, I find price-per-token to still be a better predictor of cost.
It’s still damned capable though, I’ve done a crazy amount of good work with 5.3 Flash.
+1. FWIW, I didn't even get far enough to wonder about the price.
The lead (and only) sentence states "Our next era of frontier intelligence, built to solve your hardest problems" a statement equally true of any new Google or competitor model, hence completely devoid of useful meaning. I then kept scrolling down... and down... and down, giving up after a half dozen screens of big images, huge videos and some customer stories. The first sentence of every text block was equally vague or meaningless so... tv;dr (Too vague, didn't read).
Even the Gemini sub-reddit is laughing: https://www.reddit.com/r/GeminiAI/comments/1x2dr47/the_lates...
It was the case a year ago, but it's less clear cut nowadays, I've seen non-tech people using claude on their phone in the train, and many tech people seem to have switched from Claude Code + Opus to Codex + Sol over the past few month.
It might not be as smart, though my employer gives me access to Claude, so I tried giving the same task to Opus 5.5. Both DS4.1 Flash and Opus5.5 produced nearly identical outputs.
After that, my company switched the entire engineering department to Azure hosted DeepSeek4.1 and costs were cut by 3/4. So far, no one has really noticed the change.
https://imgur.com/a/fWo92RI
Seems to be just deleting anything that a safety filter triggers on. I don't know if I like this more or less than what Anthropic does where they train the model to actively refuse.
https://antigravity.google/download
Excellent model. Same price and score as Opus 5.5 in our 23 question exam.
This size / class of model is now a bit overpowered for these short-horizon data analytics tasks. OpenAI 6.1-Sol, Haiku 5.5, GLM 5.3 Flash all achieve about the same results but for ~1/10th the cost.
In other words, we're getting to the stage where models are "good enough" for certain tasks like data analytics and competitiveness is going towards size & speed.
Language info: https://killswitch-lang.org/
Now that needed to be done for some time!
Regardless of if an AI helped or not, regardless of whether that was Google's or some other AI or not (though Argon does sound pretty cool!), regardless of whether an AI cut the time from months to weeks or not, refactoring Chrome -- decomposing it into individual clearly demarcated non-tightly coupled software parts with core webpage handling at the center -- that needed to be done for some time!
Now all of that being said, Gemini 4 Argon does sound pretty cool, and we congratulate both teams at Google -- the team that created Gemini 4 Argon, and the team that refactored Chrome with it!
But all of that being said, while the advancements made for Gemini 4 Argon are important and impressive (and they are!)... refactoring Chrome properly (IMHO) is slightly more important and impressive!
(You know, one man's humble opinion! That is, I am not the guy who is responsible for awarding performance-based bonuses at Google at the end of the year, so take everything I say with the proverbial "grain of salt"! :-) I am probably biased, one way or the other! :-) )
"Across Google", wonder how much that is because of an effort by Google to curb usage of Anthropic models. MSFT and Meta similarly are pushing for in-house models. It doesn't mean the models are better when you push your engineers to dogfood and restrict access to other LLMs.
(Not that I know for sure this is the case, it's more of a hypothesis).
Unavailable · Gemini 4 Argon is currently available to Ultra plan subscribers.
Like when Miller in The Expanse talk to phone AI (https://m.youtube.com/watch?v=SC3bpJmGnPo) to trace Anubis flight, notice how the phone doesnt even reply in words.
Imagine if the phone instead replied with: "Thats a great insight, Anubis really goes there", it would be a joke. We are only tolerating it now because everyone is doing it.
But on December 1st Argon doubles in price - so it will be the same price as Opus 5.5.
Likewise, you look at Artificial Analysis' general index and it cost $8,708 to run with Opus and $2,407 to run with Argon so even if Argon's token price doubles, it won't be the same price as Opus.
Sonnet is cheaper than Opus per-token, but in their coding tests Sonnet burns through enough tokens that it's 7.5% more expensive.
So much depends on how token efficient something is and whether it gets you a good result or not, right?
Plus, if one is using a plan instead of paying token rates, what does my $200 get me from OpenAI, Anthropic, or Google?
It just feels very hard to compare pricing given that the number of tokens used can vary so much. In Artificial Analysis' coding tests, Sonnet used 7.1x more tokens than Sol. Sonnet and Sol might have the same per-token pricing, but there's a huge difference in the number of tokens used.
This was hardly a consideration a year ago, when many model tasks could be tested using a single prompt and it was hard to spend more than a dollar.
From what I saw online.
Edit: noble gases not noble taxes, though the latter would be a funnier naming scheme.
It feels so complicated and hard to know what we can use vs what hands them our codebases
Is it actually available?
I'd be curious to see if people have managed to just add custom instructions to make the outputs more standardized and less annoying. I never have such annoying cases, whatever the model, it's just too verbose, but in this case I ask it to be more concise and it's usually fine.
Anthropic’s vision of “safe” is so frustrating to deal with, it makes me want to go back to ChatGPT.
The code is bad though.
They all ramble, they're all allergic to getting to the point. But, Gemini historically has pretty good prose. Even the self-hosted Gemma models have pretty nice, inoffensive prose. I wouldn't want to read a novel written by any of them, but I tend to ask Gemini questions in chat more often than Claude or GPT, because it has good prose and is good at using the web to research.
Or else it say, here is the result. (but no it didn't do no research, and tells me, sorry I forgot to search).
It's got balls to lie straight at my face.
build this scene space station docking scene https://x.com/_satyam_ai/status/2110097633922662752?s=20
I'm starting to feel the subscription fatigue and a small amount of FOMO. I just can't justify pro accounts for every frontier model now that it includes OpenAI, Anthropic, Meta and Google.
https://youtu.be/DlTNN0gvkLM
Gemini 3 Flash was around 0.403, 2.5 Pro was around 0.34.
I never touch 3.8, it's awful. Curious to try this, but I've yet to use any Gemini that is good for agentic work.
Also, unrelated but this made me look at Artificial Analysis to see where it lands on Cost per Task, and it ranks Opus 5.5 at almost double Astra 6. That does not align with my experience.
Every time I try to generate one it's like a slot machine, where a different part will be messed up after fixing the first.
I told it to strip screenshots from our old basic explainer video to generate modern 3D video.
Pretty happy with the result. I told it to keep the audio and match the video and it did.
I previously tried with Seedance 2.5, Flux 3, Kling 3, Gemini Omni without much luck.
I wonder if any of the AI safety people will begin encouraging AI researchers to swap jobs to work at a failing lab in order to keep them all extremely competitive with one another.
At this point a lot of stuff happens at the product level.
OpenAI and anthropic have distribution and data advantages now that are hard to replicate.
It still thinks Charlie Kirk is alive, though.
Google...why. You keep getting so close but constantly fumble at the last moment. I can use Opus 5.5 and Sol/Astra on the $20 Anthropic and OpenAI plans.
You are already playing from behind. You need to GET YOUR MODELS INTO MORE PEOPLES HANDS by any means possible. Continuing to make it difficult to use your products DOES NOT HELP.
This model might be great but the majority aren't going to subscribe for $100/month just to try it.
God knows why the server different things to the same people on different interfaces
What about the Google One AI plans? Will they get it? That has a Pro AI plan as well.
"Hans, we're doing a new point release! What isn't crossed out on the periodic table yet?"
On safari the page doesn't scroll very well on the first section
Which one is it?
e.g.
The first two prompts I tested in AI Studio does the same, unfortunately.Another quite unpleasant tick was that if you specify a peculiar term in your prompt as a conversational shortcut, it would keep repeating that throughout the conversation. e.g. "...What is a bitter lesson pilled way to do this?", then you'd see it will quote "bitter lesson pilled" at like 7 places in its response. This one seems to have eased off a little.
That was a long wait.
https://playblueline.pages.dev/
[1] https://github.com/srw/sha256-19step-preimages-with-padding
[2] https://github.com/srw/sha256-20step-preimages-with-no-paddi...
What’s the point lol. I’m not subbing to ultra to have it
I asked for a QUICK healthcheck on the sync status of a bigquery integration. It literally spun for 25m before I killed it.
The task, which is CLEARLY outlined in the agent readmes just involved using the gcloud cli to initiate auth, then just run a few queries to check logs and timestamps.
Even 3.6 flash can get this right (most the time).
OpenAI release any time now
I have found that Gemini often claims it did something, but didn't actually complete the task.
I just had 3.8 Flash do a major refactor -- that never should have needed to be done in the first place -- and switched to Argon 4, and had it review it.
Almost everything had been stubbed out by 3.8, and none of it was properly integrated into the codebase. :facepalm:
fyi: argon is an interesting choice of name, coming from the Greek argos meaning "lazy or living w/out labor".
Without going into details, for a small part of a larger task, I asked it to classify different content types of different files and formulate a structured mapping (e.g., in JSON) between content types and another set of files (deliberately left vague here but not to Argon) from my codebase. I even offered it a multi-part draft of a classification system to start with and told it that if it can find other classification systems from international standards, academia or industry more suitable to the task than my rough draft, we can discuss replacing my suggestion. There were other parts to the instructions too, written out like an article with bulleted lists, precise program-like wording, etc. Then, I sent the prompt and left for a while.
When I came back, it was nearly done with the implementation but the classification system it came up with was a mess. It even admitted to making up "bogus" classification standards when I pointed out some errors. It also kept leaving gaps multiple times when I referenced errors to fix in subsequent instructions (and not once apologized the entire time lol). So my first impression was subpar, not really comparable IMO with Opus >=5 or Sol >=6, and seemed to leave larger gaps than 3.8 flash typically does. The most irritating part to me was not only the fact that it was making up things, but that it ignored my instructions to discuss things with me before deciding to shift away from my draft; I don't recall any major LLM ignoring an instruction like that in at least a year.
The fact that multiple companies can breach any benchmark means there is no ambiguity in terms of how to build AI, its just a matter of time.
Certainly Meta and Google are benchmaxxing to please public markets, but it still proves the above
A more intelligent model would not default to spaghetti throwing, it would pause to consider the problem fundamentals, the accuracy of assumptions, and ask questions that remove ambiguity. This however does not lend itself to increasing token sales while prices remain inflated from market and brand dominance.
I wonder how much is actual improvements in the model and how much comes from better harnesses. I'd be willing to bet that the combination of better harnesses + the models being trained around agentic workflows (i.e the harnesses) has made it significantly better in many things, but it's hard to find other improvements imo.
Start preparing Gemma 5 release.
Gemini Live vs. Gemini Lite vs. Gemini Pro vs. Gemini Flash vs. Nano Banana Lite (that one apparently has no Gemini prefix?)... Please, just pick a metaphor and stick with it without wildly mixing reasoning modes, model sizes, and input/output modalities in the model name!
Source? Their documentation says it's available on the $20 plan and I can confirm firsthand.
https://one.google.com/intl/en_us/about/google-ai-plans/
https://x.com/OfficialLoganK/status/2110038940841545827
it needs MCP asap
Still reading up on it here: https://ai.google.dev/gemini-api/docs/long-decode and needs lots of community testing to determine whether it's a "true" 1M token output window.
Like a working scroll on a webpage.
God damn this is impossible to use of safari mobile.
Argon is trying to solve the "hard" problems like those.
Now, I happen to believe that's an excellent strategy. I bet they actually resolve more LLM requests than any other company at this point. The lay person still uses Google exclusively.
if they throw in youtube premium in that package then I might reconisder
they gave away so much pro that they need to have smaller usage levels than competitors or they'll lose a ton of money. $20 was already a loss for users actually using their quota. the majority of pro users are paying far less than that.
so they'll lose their paying users, and keep their free or subsudized, which will make the economics even worse.
I use the hell out of both. Kind of sad that they took Spark out of beta-access, being able to orchestrate via your phone on something that has direct access to your Drive meant I was authoring skills and other functionality on a schedule with spreadsheets serving as databases, deterministic tests, etc.
Hope they roll it out to a full feature soon
More specifically, who are they targeting with Argon and why is it behind the Ultra plan? Who is paying $99/month if not developers?
There is another subset in the Google Workspace for enterprise clients.
Corporate companies. Bog-standard office workers don't need top-of-the-line Mythos intelligence, just something to add to Google Workspace. I personally doubt developers use Gemini for coding beyond marketing materials.
Because if it’s not, and it’s not available to all paying users (Right now, it’s only available to Ultra users), then they are behind the curve if they’re treating it like Fable.
It’s tone deaf.
Whether it's actually better than OAI/Anthropic offerings, probably not, but if Google has managed to get back to parity, that's good for competition, prices, and increased model performance in the future. Three strong western competitors is better than two.
Unique users, maybe. But a simple query that's answered in a paragraph is negligible compared to devs running agents continuously for tens of hours.
When I see non technical people use that Google reply I'd be surprised if it's more than 1k output tokens, and with a tiny model. So a 3000x in output tokens, on a model at least 10x as expensive, I wouldn't be surprised if hardware usage is 30.000x for a dev that uses coding subs vs someone who uses the Google AI overview a couple of times a day.
And that's not counting the absurd difference in cache reads.
> Gemini 4 Argon (Medium)
edit: It failed on the second prompt and they disabled it now.
https://antigravity.google/blog/gemini-4-argon-in-google-ant...
> Gemini 4 Argon is available today for Google AI Pro and Ultra subscribers
1. It's available to Pro in Gemini app, but NOT antigravity (for coding)
2. It's a phased rollout and Ultra got it first, Pro is coming later
The latter blog post is specifically "Argon in Antigravity" versus the other blog post is about Argon in general, so seems like #1 is more likely.
EDIT:
Apparently Ultra subscribers gets Argon everywhere including in Google Antigravity
Pro subscribers only get them in Gemini app and Search
https://one.google.com/about/google-ai-plans/?icid=microsite...
It seems to be rolling out gradually.. not available at once to everyone.
It's crazy to me how care there is in most of these vibe-coded vibe-coding harnesses. I don't like oai's models much but I have to say their UI has by far the best attention to detail of any major player.
It doesn't show in the desktop UI.
I like this phrasing and am curious what new word or phrase we might use to describe big companies over pushing Ai internally at the expense of their product quality. This pattern is so ubiquitous across google cloud products I have taken to calling them the 21st century IBM, having been a IBM baby and witnessing first hand their fall from exceptionalism.
And took similar companies that were quality engineering focused -- like DEC or Palm -- down with them through acquisition.
This is in the Gemini app on macOS.
If they don't want to make it available even for small tests for free, that's fine, but then don't put a button suggesting I can try it without qualifying it...
EDIT: It's actually worse: The page then shows "Look what you can do with Gemini 4" and a "Let's go" button that leads to a bunch of links to examples, all with enticing pictures. Except if I click on them I get a box stat strongly suggests it'll run with Gemini 4, except the dropdown shows "Flash" and if you click on it, it shows as Gemini 3.6 again, and if you run it, the results unsurprisingly end up looking nothing like their pictures.
I don't know if that page was built by humans or AI, but clearly nobody tested it properly.
I really can’t believe how bad the Gemini experience is. I have personal paid workspace account, and meant to have some better AI Google features I think.
So, I connected my Gmail to Gemini. 3.6 Flash kept telling me it doesn’t have any Gmail connnection, and then proceeded to hallucinate instructions for me to enable it, all bogus. At least 3.1 Pro realized that my Gmail is connected and then searched.
...or Gemini 4 is being gradually rolled out and is not available to everyone all at once.
But you don’t create a broken or confusing experience for the people who haven’t been added to the rollout group yet.
It's an unfortunate downside to conflicting requirements for release-engineering and marketing urgency. In a perfect world, rollout would go from 0-100% instantly, coinciding with the release of marketing material.
In the real world, the trade-offs are either the marketing lands first for some/most people, or delaying marketing is until after rollout reaches, 100%, at which point you're scooped by tech blogs, Twitter AI influencers, and regular media, and immediately lose framing the release from the jump.
It is just sheer incompetence that this isn't near the top of their QA checklist.
Heck, even just a message saying it may not be available for all users yet would go a long way. Instead the way it currently works a lot of people will think they're testing Gemini 4 while they're in fact using Gemini 3.6 Flash and getting awful results.
Ah, that's so simple and elegant since the state is in one place -the page - and not in multiple sub-systems and products! Thank you for clearing that up.
?
Related funny https://www.youtube.com/watch?v=DlTNN0gvkLM
Which is even weirder given that the api costs are less than for the other providers
Maybe they don't really care about antigravity users. Also, monitoring usage is horrible.
Me: Gemini, navigate to nearest McDonalds.
Gemini: [Lists 10 McDonalds locations wnd asks which I want.]
Me: What were the priginal instructions just now?
Gemini: [Recites imperatives about tone and content from its memory system.]
Me: No. I told you to navigate to the NEAREST McDonalds. There's only one NEAREST McDonalds. Do that one.
Gemini: My mistake, navigating to the nearest McDonalds. [Opens Maps, navigating to a McDonalds across town.]
I also have had shockingly bad results with gemini on android. Ended up giving up on it after a few months. Silly silly google with their trademark UX polish and attention to usability.
Flash lite is very noticeably more idiotic than 3.8 Flash. It's why the quick AI overview is often much worse than the AI mode one.
The only thing I can figure is that Google's consumer-facing free-to-use models are set to "reason as fast as possible with only the most basic plausibility checks." I have to correct them like this every single time.
Similarly, you can run local LLMs on your Google Pixel smartphone, but if you want them to be hardware accelerated using the chip that's included for that very purpose on your phone... it has to be Google's models via Google's apps. So you can't say "alright, a fast, local model which sounds great, but I want one that is trained to always run a search before answering, in a harness with good search tools for answering the kinds of practical questions I often ask", you're actually not free to put that together in a practicable way.
So there is indeed some competition here. But it's not really fair competition.
1. I ask it to list out all my old Keep notes that are just throwaway task lists, reminders, etc. to propose deleting them.
2. It comes up with like 4 of them (I know for a fact there are dozens, if not hundreds). I ask it for an exhaustive list.
3. I see its thinking trace briefly show "Refusing to produce an exhaustive list..." and then it says "Yes, that is all of them."
Not sure whether it's lack of intelligence, bad tools, or both, but that was extremely frustrating to see.
I think these companies have good models but expose them poorly like others have mentioned
If it had exact location access people would be crying that Gemini steals their location data.
I tell it to "Play Let it go from frozen". It plays it on youtube music. 5 seconds later I tell it again 'Play let it go from frozen" it then just dies saying "I can't play videos in car".
i benchmarked 20+ models in real worlds and gemini families are the best (highest customer satisfaction)
The others are quite bad as well in that regard, but on this particular metric, Google has always been top dog. Goes back to way before AI ever took off, BTW.