Nope, and here's why. The Chinese model I'm talking about runs locally on my own hardware. No network, nothing to phone home with. That's the whole appeal of open weights, you can literally air-gap them. A hosted API can't offer that no matter whose flag is on it.
And funny thing, the "safe American" option isn't as clean as people assume. To use Anthropic's own top models on Bedrock, Mythos and Fable, you have to opt into 30 days of your prompts and outputs being retained, plus human review. Your data leaves AWS and goes to Anthropic, and they've said future models this powerful come with the same string attached. So it's not really Chinese-scary-model vs American-safe-model. It's my data staying on my machine vs my data landing on someone else's.
And for the record I'm not some open-source purist. I run Claude and Codex every day. I just like knowing where the door is, and honestly the harness does more of the work than the model anyway, so walking out stays cheap.
I have 48 gb ram on M4 and Gemma4 somewhat runs, but it's nowhere near as usable as Claude. Doing it myself is literally much faster than doing anything locally (yet). I also tried North Mini Code and that didn't have prompt caching so it was many times slower than Gemma4. Do you have better experience with chinese models? If yes, could you hint which?
Yeah, Qwen’s the family I run locally, and it holds up for review and refactor work. But the model’s the smaller lever here, two things moved the needle more for me than any model choice.
Start with LM Studio. You can tune context length, GPU offload, and KV cache per model, and that kills most of the slowness you’re hitting. The default context is set low and chokes on anything multi-file, so push it up before touching anything else.
The second thing is the machine itself. Running the model and doing your actual work on the same box is miserable, they fight for the same memory, and that’s a lot of why local feels slow on your M4. When I go fully self-hosted the model goes on a separate Mac Mini for that reason alone.
On which model, a dense 27b is already overkill for coding, you pay memory and speed for capacity you don’t use on most tasks. The sharper direction now is the Chinese MoE models like Qwen 3.6-35B-A3B, small active param count so they stay fast while performing near the top.
Thanks, will take a second look. I will have to go lower on the parameters count as I noticed the mem pressure got high and started swapping. I did change the context though. LM studio is like ollama but more gui oriented?
<< The structure went next, and this time the money signed the order in daylight. The nonprofit became a for-profit with the profit cap removed, and SoftBank tied tens of billions in funding to that conversion being finished on schedule. The firewall built to keep capital from steering the mission came down because capital asked for it in a term sheet. >>
That’s the only part of your excellent post that’s not true. OpenAI is still a nonprofit. It licenses its database and “science” to its affiliated for-profit, ChatGPT.
Its illegal LLM that’s built on top of IP infringement was granted permission by the IRS because it was only to be used for research, never for profit.
Thanks, and you’re basically right about the smell of it, just a year behind on the org chart. OpenAI actually flipped last October. The nonprofit didn’t stay the company, it moved up a level and became the OpenAI Foundation, and everything under it is now a for-profit, OpenAI Group PBC. The Foundation controls it and owns about a quarter of it, somewhere around $130B.
So my paragraph holds up. The profit cap really is gone, and SoftBank really did tie its money to the conversion closing on time.
But honestly your gut is pointing at the right thing. A nonprofit sitting on top of a for-profit it owns a minority of, still calling the shots, is exactly the move people are calling mission-washing. Public Citizen said the same, that the Foundation now works for the for-profit instead of the other way around, which is backwards from how it was set up. You’re aiming at the right target, just with last year’s map.
That’s weird, because the IRS Web site doesn’t show application paperwork for OpenAI Fpundation. Nor does it show “final” tax return for the original OpenAI.
There’s also an odd issue that I don’t see addressed (and I’ll search a little more tomorrow). The nonprofit may only transfer assets to another nonprofit. I’m not a tax CPA. But I’ve done the taxes for my tiny nonprofit (relatively tiny, compared to OpenAI) for 25 years. I want to hand it off in another year or two and must be super careful. I may not sell assets to an individual or to a for-profit.
If we had a semi-ethical government, I doubt Altman could get away with his sleight of hand.
You know this side better than I do, 25 years of nonprofit filings versus my zero. So I'll defer to your gut here. The part I do know is that the assets didn't leave the way you'd expect. The nonprofit didn't dissolve, it stayed on top as the Foundation and kept a stake, which is probably why you're not finding a final return for the old entity. Whether that's a clean way around the "charitable assets stay charitable" rule or a dressed-up dodge is exactly the question the AGs got asked, and a bunch of nonprofit lawyers think it's the second one. You're in better company than you think.
Saw this one after the other! 🌻It didn’t dissolve. Right!
But this is where Altman et al have a problem. And I remember when they formed ChatGPT and leased the LLM to them, there was a discussion about how much ChatGPT must be paying tax attorneys.
When the IRS functions properly, assets of a nonprofit are closely guarded to ensure that there’s not a whiff of impropriety. When I set up my nonprofit, I wanted the print side (Gestapo interrogation transcripts etc) to be part of the nonprofit. Nope. Not allowed. I therefore kept the underlying assets—archives, database, translations, photographs, videography—in my personal name, since I’d bought them personally.
Since the IRS was so insistent about the print/publishing aspect, I was equally insistent on retaining my intellectual property. Got it in writing from the IRS, so there’s no question going forward.
OpenAI’s setup put all those things in the nonprofit. Since they put the programmers’ salaries in OpenAI (capitalized software expenditures), the work performed by the programmers also is IP of OpenAI. And licensing to ChatGPT has to be at third party, fair market value rates.
Hahahahaha.
As an outsider looking in, it seems to be a massive house of cards.
If you go sulk in a corner somewhere, take is with you.
Ha, the sulk travels well, it's open weights, I can run it anywhere :)
Do you worry about the Chinese open weight models doing bad things with your data?
Nope, and here's why. The Chinese model I'm talking about runs locally on my own hardware. No network, nothing to phone home with. That's the whole appeal of open weights, you can literally air-gap them. A hosted API can't offer that no matter whose flag is on it.
And funny thing, the "safe American" option isn't as clean as people assume. To use Anthropic's own top models on Bedrock, Mythos and Fable, you have to opt into 30 days of your prompts and outputs being retained, plus human review. Your data leaves AWS and goes to Anthropic, and they've said future models this powerful come with the same string attached. So it's not really Chinese-scary-model vs American-safe-model. It's my data staying on my machine vs my data landing on someone else's.
And for the record I'm not some open-source purist. I run Claude and Codex every day. I just like knowing where the door is, and honestly the harness does more of the work than the model anyway, so walking out stays cheap.
Interesting. So completely airbases. Not even access to GitHub…. I need to grok how I can get this to work in my setup.
https://youtu.be/UngVdAsQEiU But, even 36gb of ram is tight enough
I have 48 gb ram on M4 and Gemma4 somewhat runs, but it's nowhere near as usable as Claude. Doing it myself is literally much faster than doing anything locally (yet). I also tried North Mini Code and that didn't have prompt caching so it was many times slower than Gemma4. Do you have better experience with chinese models? If yes, could you hint which?
Yeah, Qwen’s the family I run locally, and it holds up for review and refactor work. But the model’s the smaller lever here, two things moved the needle more for me than any model choice.
Start with LM Studio. You can tune context length, GPU offload, and KV cache per model, and that kills most of the slowness you’re hitting. The default context is set low and chokes on anything multi-file, so push it up before touching anything else.
The second thing is the machine itself. Running the model and doing your actual work on the same box is miserable, they fight for the same memory, and that’s a lot of why local feels slow on your M4. When I go fully self-hosted the model goes on a separate Mac Mini for that reason alone.
On which model, a dense 27b is already overkill for coding, you pay memory and speed for capacity you don’t use on most tasks. The sharper direction now is the Chinese MoE models like Qwen 3.6-35B-A3B, small active param count so they stay fast while performing near the top.
Thanks, will take a second look. I will have to go lower on the parameters count as I noticed the mem pressure got high and started swapping. I did change the context though. LM studio is like ollama but more gui oriented?
<< The structure went next, and this time the money signed the order in daylight. The nonprofit became a for-profit with the profit cap removed, and SoftBank tied tens of billions in funding to that conversion being finished on schedule. The firewall built to keep capital from steering the mission came down because capital asked for it in a term sheet. >>
That’s the only part of your excellent post that’s not true. OpenAI is still a nonprofit. It licenses its database and “science” to its affiliated for-profit, ChatGPT.
Its illegal LLM that’s built on top of IP infringement was granted permission by the IRS because it was only to be used for research, never for profit.
If that’s not fraud, I don’t know what is.
Thanks, and you’re basically right about the smell of it, just a year behind on the org chart. OpenAI actually flipped last October. The nonprofit didn’t stay the company, it moved up a level and became the OpenAI Foundation, and everything under it is now a for-profit, OpenAI Group PBC. The Foundation controls it and owns about a quarter of it, somewhere around $130B.
So my paragraph holds up. The profit cap really is gone, and SoftBank really did tie its money to the conversion closing on time.
But honestly your gut is pointing at the right thing. A nonprofit sitting on top of a for-profit it owns a minority of, still calling the shots, is exactly the move people are calling mission-washing. Public Citizen said the same, that the Foundation now works for the for-profit instead of the other way around, which is backwards from how it was set up. You’re aiming at the right target, just with last year’s map.
That’s weird, because the IRS Web site doesn’t show application paperwork for OpenAI Fpundation. Nor does it show “final” tax return for the original OpenAI.
There’s also an odd issue that I don’t see addressed (and I’ll search a little more tomorrow). The nonprofit may only transfer assets to another nonprofit. I’m not a tax CPA. But I’ve done the taxes for my tiny nonprofit (relatively tiny, compared to OpenAI) for 25 years. I want to hand it off in another year or two and must be super careful. I may not sell assets to an individual or to a for-profit.
If we had a semi-ethical government, I doubt Altman could get away with his sleight of hand.
You know this side better than I do, 25 years of nonprofit filings versus my zero. So I'll defer to your gut here. The part I do know is that the assets didn't leave the way you'd expect. The nonprofit didn't dissolve, it stayed on top as the Foundation and kept a stake, which is probably why you're not finding a final return for the old entity. Whether that's a clean way around the "charitable assets stay charitable" rule or a dressed-up dodge is exactly the question the AGs got asked, and a bunch of nonprofit lawyers think it's the second one. You're in better company than you think.
Saw this one after the other! 🌻It didn’t dissolve. Right!
But this is where Altman et al have a problem. And I remember when they formed ChatGPT and leased the LLM to them, there was a discussion about how much ChatGPT must be paying tax attorneys.
When the IRS functions properly, assets of a nonprofit are closely guarded to ensure that there’s not a whiff of impropriety. When I set up my nonprofit, I wanted the print side (Gestapo interrogation transcripts etc) to be part of the nonprofit. Nope. Not allowed. I therefore kept the underlying assets—archives, database, translations, photographs, videography—in my personal name, since I’d bought them personally.
Since the IRS was so insistent about the print/publishing aspect, I was equally insistent on retaining my intellectual property. Got it in writing from the IRS, so there’s no question going forward.
OpenAI’s setup put all those things in the nonprofit. Since they put the programmers’ salaries in OpenAI (capitalized software expenditures), the work performed by the programmers also is IP of OpenAI. And licensing to ChatGPT has to be at third party, fair market value rates.
Hahahahaha.
As an outsider looking in, it seems to be a massive house of cards.
Go here and search for OpenAI. You can see their filing documents, their tax returns… https://apps.irs.gov/app/eos/details/