New interpreter --os powered by Anthropic

.gitignore

@@ -235,4 +235,9 @@ misc/
 
 # Ignore litellm_uuid.txt
 litellm_uuid.txt
+
+# some more
 .aider*
+file.txt
+numbers.txt
+poetry.lock

examples/screenpipe.ipynb

@@ -0,0 +1,175 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Open Interpreter and ScreenPipe Cookbook\n",
+    "\n",
+    "This cookbook explores the powerful combination of Open Interpreter and ScreenPipe, two tools that can significantly enhance your ability to interact with and process digital information. Open Interpreter allows you to execute natural language commands, while ScreenPipe captures and analyzes screen content and audio output from your computer."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Prerequisites\n",
+    "\n",
+    "Before we begin, make sure you have installed the following:\n",
+    "\n",
+    "1. [Open Interpreter](https://docs.openinterpreter.com/getting-started/introduction)\n",
+    "2. [Screenpipe CLI](https://docs.screenpi.pe/docs/getting-started#cli-installation)\n",
+    "\n",
+    "Make sure both are properly installed and configured on your system."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Setting Up Open Interpreter"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Import necessary libraries\n",
+    "from interpreter import interpreter\n",
+    "from datetime import datetime, timezone\n",
+    "\n",
+    "# Configure Open Interpreter\n",
+    "interpreter.llm.model = \"groq/llama-3.1-70b-versatile\"\n",
+    "interpreter.computer.import_computer_api = False\n",
+    "interpreter.llm.supports_functions = False\n",
+    "interpreter.llm.supports_vision = False\n",
+    "interpreter.llm.context_window = 100000\n",
+    "interpreter.llm.max_tokens = 4096\n",
+    "\n",
+    "# Add the current date and time in UTC\n",
+    "current_datetime = datetime.now(timezone.utc).strftime(\"%Y-%m-%d %H:%M:%S UTC\")\n",
+    "print(f\"Current date and time: {current_datetime}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Defining the ScreenPipe Search Function"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Define the custom ScreenPipe search function\n",
+    "custom_tool = \"\"\"\n",
+    "import requests\n",
+    "import json\n",
+    "from urllib.parse import quote\n",
+    "\n",
+    "def search_screenpipe(query, limit=5, start_time=None, end_time=None):\n",
+    "    base_url = f\"http://localhost:3030/search?q={quote(query)}&content_type=ocr&limit={limit}\"\n",
+    "    \n",
+    "    if start_time:\n",
+    "        base_url += f\"&start_time={quote(start_time)}\"\n",
+    "    if end_time:\n",
+    "        base_url += f\"&end_time={quote(end_time)}\"\n",
+    "    \n",
+    "    response = requests.get(base_url)\n",
+    "    if response.status_code == 200:\n",
+    "        data = response.json()\n",
+    "        # Remove duplicates based on text content\n",
+    "        unique_results = []\n",
+    "        seen_texts = set()\n",
+    "        for item in data[\"data\"]:\n",
+    "            text = item[\"content\"][\"text\"]\n",
+    "            if text not in seen_texts:\n",
+    "                unique_results.append(item)\n",
+    "                seen_texts.add(text)\n",
+    "        return unique_results\n",
+    "    else:\n",
+    "        return f\"Error: Unable to fetch data from ScreenPipe. Status code: {response.status_code}\"\n",
+    "\"\"\"\n",
+    "\n",
+    "# Add the custom tool to the interpreter's environment\n",
+    "interpreter.computer.run(\"python\", custom_tool)\n",
+    "print(\"ScreenPipe search function defined and added to the interpreter's environment.\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Setting Custom Instructions for Open Interpreter"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Set custom instructions for Open Interpreter\n",
+    "interpreter.custom_instructions = f\"\"\"\n",
+    "Current date and time: {current_datetime}\n",
+    "\n",
+    "ScreenPipe is a powerful tool that continuously captures and indexes the content displayed on your screen. It creates a searchable history of everything you've seen or interacted with on your computer. This includes text from websites, documents, applications, and even images (through OCR).\n",
+    "\n",
+    "You have access to this wealth of information through the `search_screenpipe(query, limit=5, start_time=None, end_time=None)` function. This allows you to provide more contextual and personalized assistance based on the user's recent activities and viewed content.\n",
+    "\n",
+    "The `search_screenpipe` function supports optional `start_time` and `end_time` parameters to narrow down the search to a specific time range. The time format should be ISO 8601, like this: \"2024-10-16T12:00:00Z\".\n",
+    "\n",
+    "Here's why querying ScreenPipe is valuable:\n",
+    "1. Context Recall: Users often refer to things they've seen recently but may not remember the exact details. ScreenPipe can help recall this information.\n",
+    "2. Information Verification: You can cross-reference user claims or questions with actual content they've viewed.\n",
+    "3. Personalized Assistance: By knowing what the user has been working on or researching, you can provide more relevant advice and suggestions.\n",
+    "4. Productivity Enhancement: You can help users quickly locate information they've seen before but can't remember where.\n",
+    "\n",
+    "Use the `search_screenpipe()` function when:\n",
+    "- The user asks about something they've seen or read recently.\n",
+    "- You need to verify or expand on information the user mentions.\n",
+    "- You want to provide context-aware suggestions or assistance.\n",
+    "- The user is trying to recall specific details from their recent computer usage.\n",
+    "- The user wants to search within a specific time range.\n",
+    "\n",
+    "Here's how to use it effectively:\n",
+    "1. When a user's query relates to recent activities or viewed content, identify key terms for the search.\n",
+    "2. If the user specifies a time range, use the `start_time` and `end_time` parameters.\n",
+    "3. Call the `search_screenpipe()` function with these parameters.\n",
+    "4. Analyze the results to find relevant information.\n",
+    "5. Summarize the findings for the user, mentioning the source (app name, window name) and when it was seen (timestamp).\n",
+    "\n",
+    "Remember to use this tool proactively when you think it might help answer the user's question, even if they don't explicitly mention ScreenPipe.\n",
+    "\"\"\"\n",
+    "\n",
+    "print(\"Custom instructions set for Open Interpreter.\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

interpreter/__init__.py

@@ -1,3 +1,59 @@
+import sys
+
+if "--os" in sys.argv:
+    from rich import print as rich_print
+    from rich.markdown import Markdown
+    from rich.rule import Rule
+
+    def print_markdown(message):
+        """
+        Display markdown message. Works with multiline strings with lots of indentation.
+        Will automatically make single line > tags beautiful.
+        """
+
+        for line in message.split("\n"):
+            line = line.strip()
+            if line == "":
+                print("")
+            elif line == "---":
+                rich_print(Rule(style="white"))
+            else:
+                try:
+                    rich_print(Markdown(line))
+                except UnicodeEncodeError as e:
+                    # Replace the problematic character or handle the error as needed
+                    print("Error displaying line:", line)
+
+        if "\n" not in message and message.startswith(">"):
+            # Aesthetic choice. For these tags, they need a space below them
+            print("")
+
+    import pkg_resources
+    import requests
+    from packaging import version
+
+    def check_for_update():
+        # Fetch the latest version from the PyPI API
+        response = requests.get(f"https://pypi.org/pypi/open-interpreter/json")
+        latest_version = response.json()["info"]["version"]
+
+        # Get the current version using pkg_resources
+        current_version = pkg_resources.get_distribution("open-interpreter").version
+
+        return version.parse(latest_version) > version.parse(current_version)
+
+    if check_for_update():
+        print_markdown(
+            "> **A new version of Open Interpreter is available.**\n>Please run: `pip install --upgrade open-interpreter`\n\n---"
+        )
+
+    if "--voice" in sys.argv:
+        print("Coming soon...")
+    from .computer_use.loop import run_async_main
+
+    run_async_main()
+    exit()
+
 from .core.async_core import AsyncInterpreter
 from .core.computer.terminal.base_language import BaseLanguage
 from .core.core import OpenInterpreter