Allow the use of absolute paths for loras and embeddings

[rmatif]

This PR adds support for using absolute paths for loras and embeddings. When an absolute path is detected, it overrides --lora-model-dir and --embed-dir

Additionally, a new tag has been added to improve detection and handling of embeddings. Previously the code was case-sensitive to filenames and failed to load models containing uppercase characters

This feature is especially useful when managing many loras and embeddings spread across multiple folders and disks

Example usage:

./build/bin/sd -m path/to/model/model.safetensors -p "<embed:/path/to/embed/embed1.safetensors>, cute cat, <embed:/path/to/embed/embed2.safetensors> <lora:/path/to/lora/lora.safetensors:1>"

[wbruna]

Note this could have security implications: for instance, for web servers that call the library directly. It may be better to explicitly enable this support through a context flag, even if it ends up fixed as true on main.cpp.

Another possibility could be turning lora-model-dir and embed-dir into lists, and check any absolute path against those. The user could then pass / as a directory to effectively turn off the validation.

[stduhpf]

Note this could have security implications: for instance, for web servers that call the library directly. It may be better to explicitly enable this support through a context flag, even if it ends up fixed as true on main.cpp.

The current implementation already has a similar problem, since it can parse arbitrary relative paths (allowing to escape the lora model dir with <lora:../other-dir/[...]:1> syntax). I really think this whole thing should be reworked so that the lora paths in the prompt are parsed in the cli example (main.cpp) rather than the library.

[rmatif]

Note this could have security implications: for instance, for web servers that call the library directly

You just filter the prompt on the server side to remove the tags, and you stay safe

Another possibility could be turning lora-model-dir and embed-dir into lists, and check any absolute path against those. The user could then pass / as a directory to effectively turn off the validation.

I had already thought of that, but when you’re dealing with many directories and some of them have thousands of subfolders, it becomes impractical. This lazy way is the easiest

I really think this whole thing should be reworked so that the lora paths in the prompt are parsed in the cli example (main.cpp) rather than the library.

I agree but it’s more useful in the API than in main.cpp, which remains something for personal use. At a certain point we do need to make sd-server a simple http server, similar to llama-server

[stduhpf]

I agree but it’s more useful in the API than in main.cpp, which remains something for personal use.

At the very least I think the LoRAs+weights should be extracted from the prompt outside of the generate_image() function. Maybe having a function to parse the prompt and extract LoRAs can be useful to expose as part of the API, but I don't really like the way it's done right now.

Off topic, but this also makes me think having a lower-level (cpp?) API to make more complex workflows without touching the library code would be nice.