Zos_Kia

That’s a room temp take at best

But then how am I supposed to use your “research” to make imaginary claims on generational attention spans ?

I don’t remember people being offended by the word fuck in 2000. Sure on TV it could be considered dicey but on the internet it was pretty fair game

“i have collected some soil samples from the mesolithic age near the Amazon basin which have high sulfur and phosphorus content compared to my other samples. What factors could contribute to this distribution?”

Haha yeah the top execs were tripping balls if they thought some off-the-shelf product would be able to answer this kind of expert questions. That’s like trying to replace an expert craftsman with a 3D printer.

What kind of use-cases was it, where you didn’t find suitable local models to work with ? I’ve found that general “chatbot” things are hit and miss but more domain-constrained tasks (such as extracting structured entities from unstructured text) are pretty reliable even on smaller models. I’m not counting my chickens yet as my dataset is still somewhat small but preliminary testing has been very promising in that regard.

Most projects I’ve been in contact with are very aware of that fact. That’s why telemetry is so big right now. Everybody is building datasets in the hopes of fine tuning smaller, cheaper models once they have enough good quality data.

I doubt these tools will ever get to a level of quality that can confuse a court. They’ll get better, sure, but they’ll never really get there.

Did you listen to that hardcore history episode? It was crazy

That’s the problem with imaginary enemies. They have to be both ridiculously incompetent, and on the verge of controlling the whole world. Sounds familiar doesn’t it?

I want to believe they’re not that common anymore but who the fuck knows honestly

Also on the Californian side you had the sugar shack where all classic rock superstars went to do the exact same thing.

We often forget that the hippie scene was predominantly white, male, and came from privileged backgrounds. That’s the only demographic that was liberated by the freedom movement. Same with the NY arty scene.

If I understand these things correctly, the context window only affects how much text the model can “keep in mind” at any one time. It should not affect task performance outside of this factor.

Yeh, i did some looking up in the meantime and indeed you’re gonna have a context size issue. That’s why it’s only summarizing the last few thousand characters of the text, that’s the size of its attention.

There are some models fine-tuned to 8K tokens context window, some even to 16K like this Mistral brew. If you have a GPU with 8G of VRAM you should be able to run it, using one of the quantized versions (Q4 or Q5 should be fine). Summarizing should still be reasonably good.

If 16k isn’t enough for you then that’s probably not something you can perform locally. However you can still run a larger model privately in the cloud. Hugging face for example allows you to rent GPUs by the minute and run inference on them, it should just net you a few dollars. As far as i know this approach should still be compatible with Open WebUI.

There are not that many use cases where fine tuning a local model will yield significantly better task performance.

My advice would be to choose a model with a large context window and just throw in the prompt the whole text you want summarized (which is basically what a rag would do anyway).

But what if that money goes to art I don’t personally like?

Good point, the health of the speculative market (“people buying and selling btc and other cryptos”) is a different thing from the health of the industry (“people starting crypto projects and raising VC money to fund them”).

By fundraising, i meant the second one. I find it to be a better indicator because it sends the message that people are building projects with promises of value convincing enough that VCs invest in them. I personally think those promises are bullshit but if VCs are pouring 10 billion bucks a year in an industry you cannot credibly call it dead or dying.

To the more general point of the meme… some people only consume headlines. Their pattern is easily recognizable :

• the mainstream (as in non-tech) news cycle has calmed down a bit so they think AI is on the brink of death
• big names like Sam Altman come across as insufferable assholes so they think that’s representative of the tens of thousands of people currently working the field
• they don’t keep up to date on the subject so they have no idea how far everything has gone in just the past 12 months
• they over-invest on marginal news like the “model collapse” paper which was reposted a million times on Lemmy and Reddit

It’s the same energy as QAnon when they were convinced that Trump would parachute from a helicopter at Biden’s investiture to commandeer the US army and publicly execute Hilary Clinton. They really do believe the “AI hype” is about to simmer down and then they’ll have been right all along lmao

commenting again cause the other poster’s remark prodded me into digging the numbers on fundraising and it’s pretty interesting :

So the best year for crypto was 2021 with >30B$ raised. 2024 is projected for 10B$ raised so indeed that’s a divide by 3, pretty grim picture, right ?

Except if you take a look across industries, it’s obvious that 2021 was an anomaly year everywhere. For example Healthtech peaked around 60B$ in 2021 before going down to 15B$ in 2024. Surely you don’t think that Healthtech is a dead fad, do you ? That pattern is consistent across industries.

You know what you’re right, i was mistakenly taking 2022 as a baseline. I’ll edit my original comment to reflect that.

I don’t think GP is arguing that crypto is a good thing here. They are refuting the meme which calls crypto a dead fad.

As mind boggling as it may sound, crypto is still a very strong industry, raising about 10 billion per year. Sure fundraising has been divided by 3 since the hype years but that’s still very comfortable numbers.

Again, not saying it is a good thing. But just because it doesn’t make mainstream headlines anymore doesn’t mean it’s dead.

It might come from non native speakers too. For example in french using aesthetic as an adjective to mean “beautiful” is correct, and it may be true in other romance languages.