I mean it in the sense that I can upload a low quality phone photo of a page from a Chinese cookbook and it will OCR it, translate it into English and give me a summary of the ingredients.
I’ve been looking into vision models but they seem daunting to set up, and the specs say stuff like 384x384 image resolution, so it doesn’t seem like it would be able to do what I look for. Am I even searching in the right direction right now?
Most frontends should have you covered and scale down the image appropriately (and automatically). I’m not entirely sure about that. I think working on resolutions higher than supported should either not work due to the image encoder, or lead to degraded performance. Usually they’re scaled to what the model needs, somewhere within the pipeline. You can crop them if you like, that sheds off some pixels. Or split them up and feed them in one part after the other, if that somehow makes sense. But I bet with most software you can just upload a random image and it’ll do whatever scaling is required for you.
Managed to run it with llama.cpp. It was a great suggestion, thank you! MiniCPM-o-2_6 iq4 managed to read text from a picture of a shirt that gemma could not get right