Add new template: LocalAI #90

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

Nish-077 wants to merge 1 commit into nosana-ci:main from Nish-077:add-template-LocalAI

Nish-077 commented Apr 15, 2025

LocalAI Template Addition

Overview

Added a new template for LocalAI server that provides OpenAI-compatible API endpoints with enterprise-grade open-source models for multiple AI tasks.

Key Features Added

Multi-modal capabilities with pre-configured models:
- Text Generation: Mixtral-8x7B-Instruct
- Image Generation: SDXL-Turbo
- Text-to-Speech: XTTS v2
- Speech Recognition: Whisper Large V3
Web UI for model management and testing
Access to 900+ open source models
Runtime model loading capabilities
Comprehensive API examples for all modalities

Technical Details

Added complete job definition with GPU support
Configured environment variables for optimal performance
Added CUDA 12.x support
Implemented proper model loading paths
Added documentation for all API endpoints

Testing

Successfully tested on RTX 3090 (24GB VRAM)
Verified all API endpoints functionality
Tested model loading and inference
Job ID for reference: 4a8d6ytFZXXPc4U9WsnPLN3FduWCvBj28Y3SbTfv1MxL

Documentation

Added comprehensive README with setup instructions
Included detailed API examples
Added model management documentation
Included system requirements and configuration details

Screenshots

Added screenshot of working interface at https://pasteboard.co/mtQDvtlashjJ.png


          Add new template: LocalAI

4fdf3a7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet