feedback
Model is pretty decent. A big issue ive been noticing is its understanding of human anatomy and sporadic chunks of nonsense text.
Examples:
Bad Understanding of Anatomy - "pulls her hand away. But not completely. Instead, she tucks it neatly into the front pocket of his jeans, her knuckles resting comfortably against his hip bone"
- Most likely if you put your hand in someones pocket your knuckles would not be resting on their hip bone. Slight chance you put your hand in backwards but within this context and the orientation of the hand is wrong.
Chunk of nonsense text:
Her gaze drops for a second, then lifts again, meeting his directly. There's a vulnerability in her stare now,
a raw honesty that goes beyond simple conversation. "Do you ever," she whispers, leaning in just the slightest bit closer,
her warm breath carrying the scent of her strawberry lip gloss, "…get tired of all the hard parts?"
More context (preceding paragraph):
"I guess… I like those too," she admits softly, her voice a breath against the volume of the film.
"But I also like the ones that aren't so… heavy." A shy smile plays on her lips, genuine and full of hopefulness.
"You know, where the good guys win and it all works out in the end." She gives a little shrug, her shoulders rising and falling.
"Sometimes… it's nice to believe in that, even if it's just for a couple of hours."
Her gaze drops for a second, then lifts again, meeting his directly. There's a vulnerability in her stare now, a raw honesty that goes beyond simple
conversation. "Do you ever," she whispers, leaning in just the slightest bit closer, her warm breath carrying the scent of her strawberry lip gloss,
"…get tired of all the hard parts?"*
- The use of the word "hard" here is the big problem. Its most likely a result of the model choosing the token that is the most sexual/suggestive as opposed to one that makes sense.
Yeah, it's hard to tell if these logic issues are due to the finetune or just air occasionally having some hiccups.
There's still something not quite right with the finetuning process on MoE's that I'm trying to troubleshoot at the moment, as I notice that making them learn the dataset with any amount of force quickly makes them go stupid, which doesn't feel right.
Most of my experience comes from voice models. What I noticed or what it appears you guys all do is you train on top of your old versions. So you would make a v3 from this v2 model. (Maybe i'm wrong, if so then disregard).
With voice models we cannot do this building on top of failed models. If the model is bad or has some type of artifact then we throw it out. I have to make sure the foundation is solid.
I think the misunderstanding arises because iteration while training models doesn't work the same way as something like software development would. Because the model retains information that we cannot specifically target and pick out. We are more blind in a sense, using trial and error and maybe some educated guessing. If for example the model struggles with directions and confuses its left from its right. We can train it on a dataset that shows clear examples of left and right and we can even guess at which layers need to be frozen or not. But this isn't nearly as precise as going into some code somewhere finding a faulty function and fixing it.