OpenAI Explorer (Experimental)

An experimental extension to use OpenAI’s services to create or manipulate geometry in SketchUp using natural language.

Description

This tool uses OpenAI’s GPT-based AI (artificial intelligence) to generate SketchUp Ruby code that – when executed – can draw geometry (e.g. with the prompt “Draw a box”) or manipulate drawn objects (e.g. with a command like “Color the objects in the selection randomly using five blue tones”). You can also use it to interact with OpenAI’s services using chat-like, text-only prompts like “Tell me a joke” or “How does the follow-me tool work in SketchUp”.

PLEASE NOTE: This is an experimental extension, which is able to fully automate SketchUp using its Ruby scripting engine (if code execution is enabled). This can even affect your entire file system because SketchUp has unrestricted, user level access to that. Use with caution and at your own risk! When in doubt, do not enable code execution in the extension’s settings (it is disabled by default).

ALSO NOTE: Any Ruby code that OpenAI’s service generates may contain errors (or not be executable code at all). Therefore, use this tool with caution and experiment with prompts if you run into errors. Sometimes it helps to simply re-word the prompt.

Usage

You can find this tool under the Extensions menu. However, your first step should be opening the settings dialog, which you can find under the Extensions > OpenAI Explorer (Experimental) > OpenAI Explorer Settings menu item:

Before you can use this tool, you must sign up for OpenAI API access and generate an API Key here. This requires you to set up an account with them and may lead to costs (after a free trial period). You then need to enter the API key in this settings dialog and you should be good to go. All of the other settings can stay as they are for now.

OpenAI Explorer Dialog

The actual AI prompt can then be found in a different dialog, which opens up when you click the Extensions > OpenAI Explorer (Experimental) > OpenAI Explorer menu item.

Starting with version 2.0, this extension features a chat-like interface where you can enter a prompt (e.g. “Draw a line” as shown in the following image) and then the full user prompt (as it was submitted to OpenAI) as well as the response are shown (and retained as long as the dialog remains open). This makes it easy to add subsequent queries. You can also use the Settings button to bring up the settings dialog and make changes on the fly (e.g. if you want to switch code execution on or off between requests). Make sure you read the Use Cases section below to understand what can be entered in this dialog.

TIP:
The input field in this dialog should work well with your operating system’s built-in speech-to-text functionality. For example, in Windows, use the WinKey + H key combination to bring up a microphone input.

Settings Dialog

This dialog provides several options to customize this extension:

System Message / Prompt Prefix: Prior to version 2.2, this text was added to the prompt before the user request. As of version 2.2, this text is being sent to OpenAI’s API as part of the system message portion of a request. This field can basically contain a set of instructions that tell the AI what to do with the actual request (and what not to do). Feel free to experiment with different system messages such as:

• “Generate only valid, self-contained SketchUp Ruby code without any method definitions.”
• “Use only SketchUp Ruby.”
• “Generate concise SketchUp Ruby code.”
• “Generate SketchUp Ruby code without any comments.”
• You can leave this empty if you just want to send the user request, e.g. for chatbot behavior.

(Chat Completion) Model: This is preset for a current, working model but you can enter a different model than the default here anytime. A list of valid models can be found in the OpenAI API documentation. I have also discussed some of those in various blog posts. Note the following:

• As of version 1.0.3, this extension uses the chat completion API, i.e. enter “gpt-3.5-turbo” (or compatible) here.
• Up to version 1.0.2, this extension used the text completion API, i.e. enter “text-davinci-003” (or compatible) here.

Max. Tokens: This number controls the length (and thereby potential cost) of the response. It is likely good to keep this number low unless you need it higher. The extension will output the exact number of tokens used for each request, which allows you to adjust this as needed.

Temperature: This parameter controls the “creativity” in the answer and therefore is more deterministic at zero and more variable (and possibly les reliable) at higher values.

Execute Code: By default this is set to not execute code, but if you want to automatically execute the generated code, then you can change that here. Two safety features have been implemented when this option is set to yes:

• Any request that contains a destructive word (“delete”, “erase”,…) will generate an additional user prompt before any code is executed.
• As of version 2.2, a system message is added that prevents code generation for any requests that involve file system access.

Use Cases

Create or Modify Geometry

Set the System Message / Prompt Prefix to “Use SketchUp Ruby“ (or similar – feel free to experiment) and the Execute Code option to Yes. Use this to draw or manipulate something based on your prompt and what you have selected in SketchUp. This lets you control SketchUp with natural language (e.g. “Draw a box”). Note that this works better with simple and specific commands than with complex concepts (that would include multiple modeling steps, such as “Draw a house”). Rather, break up complex modeling tasks into simple steps. Also, be very careful with requests that delete something.

To give you some examples, following is a list of prompts that I have used successfully:

• “Draw a box” (these often get drawn downward from the origin because the positive pushpull operation uses a face’s white face)
• “Color the items in the selection randomly in five different blue tones” (selecting something and then asking the AI to modify those seems to work very well)
• “Scale each item in the selection randomly in the z direction” (similar behavior)
• “Replace the selected line with a construction line” (works, but only on a single line)
• “Select all vertical faces in the model” (you can even have the AI select things based on a property like this)
• “Clear the selection. Then select all the components whose name includes ‘sphere'” (works best if you clear the selection first)
• “Create a new material that is aquamarine” (color materials work well but textured materials don’t as they need access to image files)
• “Add a new layer called ‘Walls'” (because the API uses “layer” instead of “tag” you will need to use the word layer here)
• “Add a new lyaer called ‘Walls'” (it understands typos correctly!)
• “Add a new scene tab with a single line of code” (adding “with a single line of code” often prevents the AI from trying to address all the options with the response, cutting down on tokens)
• “Draw a grid of 100 12 inch boxes, spaced at 24 inches” (correctly draws a 10 x 10 grid of boxes)
• “List the center points of all the objects in the selection” (this will print the coordinates to the Ruby Console using the puts command. Make sure you have the console open before you run this command)
• “List all the unique vertices in the selection” (similar)
• “Randomly erase 50% of the selected objects” (this works, but as mentioned before: use with caution)
• “Open the SketchUp website” (opens the website in a browser, same for the 3D Warehouse)
• “Zoom extents” (you can even control the view)
• “Zeichne 10 x 10 50cm Quadrate auf den Boden mit 50cm Abstand” (you can use languages other than English and various unit systems)

Generate Code for Editing

Set the System Message / Prompt Prefix to “Use SketchUp Ruby“ (or similar – feel free to experiment) and the Execute Code option to No. Use this to generate Ruby code that you can then copy and paste into your favorite code editor (e.g. my Ruby Code Editor) where you can further edit and execute it. You can use this to create and look up code snippets.

Q&A / Chat

Clear the System Message / Prompt Prefix and set the Execute Code option to No: Use this to ask the AI anything that does not necessarily generate code (e.g. “Tell me a joke”). You can this way use OpenAI’s regular chatbot behavior, e.g. to explain how something works in SketchUp.

Screenshots and Videos

Extension Terms and Conditions

Downloads and Installation

You can now find this extension in SketchUcation’s PluginStore and at the download links below. Due to Trimble’s AI-specific rules, this extension cannot be installed from the Extension Warehouse.

Alternative Downloads

Linked below are the installation files (RBZ format) for the current and some previous versions of this extension. You can also find this extension on GitHub.

• as_openaiexplorer_2-2.rbz- 2.2
• as_openaiexplorer_2-1.rbz- 2.1
• as_openaiexplorer_2-0.rbz- 2.0
• as_openaiexplorer_1-0-3.rbz- 1.0.3
• as_openaiexplorer_1-0-2.rbz- 1.0.2
• as_openaiexplorer_1-0-1.rbz- 1.0.1
• as_openaiexplorer_1-0.rbz- 1.0

Revision History

• Version 2.2 (12/9/2023):
  – Implemented correct system message handling as per OpenAI API
  – Added system message to prevent file access when code is to be executed
  – Updated default system message
  – Added menu item to reset extension settings (for troubleshooting)
• Version 2.1 (9/23/2023):
  – Better error handling
  – Fixed gpt-4 markdown extraction issue
  – Added a timer
  – Set a 30 seconds read timeout for hung requests
• Version 2.0 (7/23/2023):
  – Created dialog-based input for more chat-like experience
  – Moved error reporting into dialog to reduce pop-ups
• Version 1.0.3 (7/19/2023):
  – Added license/disclaimer file
  – Added better license display on first use
  – Added link to OpenAI TOU to menu
  – Reverted the prompt prefix to a text box (user can now enter anything)
  – Disabled code execution as the default (can be turned on in settings)
  – Changed from OpenAI completion model to chat model (OpenAI changes)
• Version 1.0.2 (3/28/2023):
  – Fixed error handling bug
  – Added a double-check for delete requests
• Version 1.0.1 (3/24/2023):
  – Added cleanup for the returned code/text
  – Added error handling for API and a one-time warning for the extension
• Version 1.0 (3/18/2023):
  – Initial release

Troubleshooting

You can reset this extension by pasting the following code into SketchUp’s Ruby Console (which can be found under the Extensions menu) and executing it (hit Enter):

Sketchup.write_default( "as_openaiexplorer" , "openai_warning" , nil )
Sketchup.write_default( "as_openaiexplorer" , "openai_explorer_settings" , nil )
Sketchup.write_default( "as_openaiexplorer" , "openai_explorer" , nil )
Sketchup.write_default( "as_openaiexplorer" , "disclaimer_acknowledged" , nil )

References

• OpenAI chat completion reference
• OpenAI chat completion API reference
• My initial post about this technology

Show Off Your Work

Did you create something amazing with this extension? I would love to see it! You can either upload an image in the comment section below (public) or privately send it to me.