Open Contracts (Demo)

The Free and Open Source Document Analytics Platform

---

Backend CI/CD
Meta

TLDR: What Does it Do?

Knowledge is power. Software is a tool. OpenContracts is FREE and OPEN SOURCE software designed to put knowledge owners and subject matter experts in charge of their knowledge. Store it in an accessible and exportable format, and make it work with emerging agentic workflows and techniques.

OpenContracts is a GPL-3.0 enterprise document analytics tool. It supports multiple formats - including PDF and txt-based formats (with more on the way). It also supports multiple document ingestion pipelines with a pluggable architecture designed to make supporting new formats and ingestion engines easy - see our Docling Integration for an example. Writing your own custom document analytics tools where the results get displayed beautifully over the original document is easy. We also support mass document data extraction with our custom LLM framework built on PydanticAI.

PDF-Annotation and Analysis:

TXT-Based Format Annotation and Analysis:

Data Extract:

Rapidly Deployable Bespoke Analytics

[DEVELOPING] Document Management

Ok, now tell me more. What Does it Do?

OpenContracts provides several key features:

1. Document Management - Organize documents into collections (Corpuses) with fine-grained permissions
2. Custom Metadata Schemas - Define structured metadata fields with validation for consistent data collection
3. Layout Parser - Automatically extracts layout features from PDFs using modern parsing pipelines
4. Automatic Vector Embeddings - Generated for uploaded documents and extracted layout blocks (powered by pgvector)
5. Pluggable Analyzer Architecture - Deploy custom microservices to analyze documents and automatically annotate them
6. Pluggable Parsing Pipelines - Support new document formats with modular parsers (Docling, NLM-Ingest, etc.)
7. Human Annotation Interface - Manually annotate documents with multi-page annotations and collaborative features
8. Custom LLM Framework - Built on PydanticAI with conversation management, structured responses, and real-time streaming
9. Bulk Data Extract - Ask multiple questions across hundreds of documents using our agent-powered querying system
10. Custom Extract Pipelines - Create bespoke data extraction workflows displayed directly in the frontend

Key Docs

We recommend you browse our docs via our Mkdocs Site. You can also view the docs in the repo:

1. Quickstart Guide - You'll probably want to get started quickly. Setting up locally should be pretty painless if you're already running Docker.
2. Basic Walkthrough - Check out the walkthrough to step through basic usage of the application for document and annotation management.
3. Metadata System - Learn how to define custom metadata schemas for your documents with comprehensive validation and type safety.
4. PDF Annotation Data Format Overview - You may be interested how we map text to PDFs visually and the underlying data format we're using.
5. Custom LLM Framework - Our PydanticAI-based framework provides document and corpus agents with conversation management, structured responses, and real-time event streaming.
6. Vector Store Architecture - We've used the latest open source tooling for vector storage in postgres to make it almost trivially easy to combine structured metadata and vector embeddings with our LLM agents.
7. Write Custom Data Extractors - Custom data extract tasks are automatically loaded and displayed on the frontend to let users select how to ask questions and extract data from documents.

Architecture and Data Flows at a Glance

Core Data Standard

The core idea here - besides providing a platform to analyze contracts - is an open and standardized architecture that makes data extremely portable. Powering this is a set of data standards to describe the text and layout blocks on a PDF page:

Modern, Pluggable Document Processing Pipeline

OpenContracts features a powerful, modular pipeline system for processing documents. The architecture supports easy creation and integration of custom parsers, embedders, and thumbnail generators:

Each pipeline component inherits from a base class that defines a clear interface:

• Parsers: Extract text and structure from documents
• Embedders: Generate vector embeddings for semantic search
• Thumbnailers: Create visual previews of documents

Learn more about:

• Pipeline Architecture Overview
• Docling Parser
• NLM-Ingest Parser

The modular design makes it easy to add custom processors - just inherit from the appropriate base class and implement the required methods. See our pipeline documentation for details on creating your own components.

Limitations

At the moment, we only support PDF and text-based formats (like plaintext and MD). With our new parsing pipeline, we can easily support other ooxml office formats like docx and xlsx, HOWEVER, open source viewers and editors are a rarity. One possible route is to leverage the many ooxml --> MD tools that now exist. This will be a reasonably good solution for the majority of documents once we add a markdown viewer and annotator (see our roadmap).

Production Deployment

For production deployments, OpenContracts includes a dedicated migration service to ensure database schema updates are applied correctly and efficiently:

Database Migrations

Before starting production services, run database migrations using the dedicated migration service:

# Run migrations first
docker compose -f production.yml --profile migrate up migrate

# Then start main services  
docker compose -f production.yml up

The migration service:

• Runs exactly once to avoid race conditions
• Uses Docker Compose profiles for isolation
• Only depends on PostgreSQL, not other services
• Ensures django_celery_beat and other app tables are created before dependent services start

This prevents issues like celerybeat failing due to missing database tables.

Acknowledgements

Special thanks to AllenAI's PAWLS project and Nlmatics nlm-ingestor. They've pioneered a number of features and flows, and we are using their code in some parts of the application.

NLmatics was also the creator of and inspiration for our data extract grid and parsing pipeline UI/UX:

The company was ahead of its time, and, while the product is no longer available, OpenContracts aims to take some of its best and most innovative features and make them open source and available to the masses!