An experimental extension to use OpenAI’s services to create or manipulate geometry in SketchUp using natural language.
Description
This tool uses OpenAI’s GPT-based AI (artificial intelligence) to generate SketchUp Ruby code that – when executed – can draw geometry (e.g. with the prompt “Draw a box”) or manipulate drawn objects (e.g. with “Color the objects in the selection randomly using five blue tones”). You can also use it to interact with OpenAI’s services using chat-like, text-only prompts like “Tell me a joke”.
PLEASE NOTE: This is an experimental extension, which is able to fully automate SketchUp using its Ruby scripting engine (if code execution is enabled). This can even affect your entire file system because SketchUp has unrestricted, user level access to that. Use with caution and at your own risk! When in doubt, do not enable code execution in the extension’s settings (it is disabled by default).
ALSO NOTE: Any Ruby code that OpenAI’s service generates may contain errors (or not be executable code at all). Therefore, use this tool with caution and experiment with prompts if you run into errors. Sometimes it helps to simply re-word the prompt.
Usage
You can find this tool under the Extensions menu. However, your first step should be opening the settings dialog, which you can find under the OpenAI Explorer (Experimental) > OpenAI Explorer Settings menu item:
Before you can use this tool, sign up for OpenAI API access and generate an API Key here. This requires you to set up an account with them and may lead to costs (after a free trial period). You then need to enter the API key in this dialog and you should be good to go. All of the other settings can stay as they are for now.
The actual AI prompt can then be found in a different dialog, which opens up when you click the OpenAI Explorer (Experimental) > OpenAI Explorer menu item.
Starting with version 2.0, this extension features a chat-like interface where you can enter a prompt (e.g. “Draw a line” in the following image) and then the full prompt (as it was submitted to OpenAI) as well as the response are shown (and retained as long as the dialog remains open). This makes it easy to add subsequent queries. You can also use the Settings button to bring up the settings dialog and make changes on the fly (e.g. if you want to switch code execution on or off between requests). Make sure you read the Use Cases section below to understand what can be entered in this dialog.
The input field in this dialog should work well with your operating system’s built-in speech-to-text functionality. For example, in Windows, use the WinKey + H key combination to bring up a microphone input.
There are several ways you can further customize the tool using with its settings. Here are your options:
(Chat Completion) Model: You can enter a different model than the default here. A list of those can be found in the OpenAI API documentation. I have also discussed some of those. Note the following:
- As of version 1.0.3, this extension uses the chat completion API, i.e. enter “gpt-3.5-turbo” (or compatible) here.
- Up to version 1.0.2, this extension uses the text completion API, i.e. enter “text-davinci-003” (or compatible) here.
Max. Tokens: This number controls the length (and thereby potential cost) of the response. It is likely good to keep this number low unless you need it higher. The extension will output the exact number of tokens used for each request (in the Ruby Console dialog), which allows you to adjust this as needed.
Temperature: This parameter controls the “creativity” in the answer and therefore is more deterministic at zero and more variable (and possibly incorrect) at higher values.
Prompt Prefix / Execute Code: The first and last options in this dialog control the request and the response. For example, if you just want to see the response and not execute the code, then you have an option here to choose that. See the Use Cases section below for more details. Feel free to experiment with different prompt prefixes, such as:
- “Use only SketchUp Ruby”
- “Generate SketchUp Ruby code without any comments”
Use Cases
Create or modify geometry
Set the Prompt Prefix to “Use SketchUp Ruby“ (or similar – feel free to experiment) and the Execute Code option to Yes. Use this to draw or manipulate something based on your prompt and what you have selected in SketchUp. This lets you control SketchUp with natural language (e.g. “Draw a box”). Note that this works better with simple and specific commands than with complex concepts (that would include multiple modeling steps, such as “Draw a house”). Rather, break up complex modeling tasks into simple steps. Also, be very careful with requests that delete something.
To give you some examples, following is a list of prompts that I have used successfully:
- “Draw a box” (these often get drawn downward from the origin because the positive pushpull operation uses a face’s white face)
- “Color the items in the selection randomly in five different blue tones” (selecting something and then asking the AI to modify those seems to work very well)
- “Scale each item in the selection randomly in the z direction” (similar behavior)
- “Select all vertical faces in the model” (you can even have the AI select things based on a property like this)
- “Clear the selection. Then select all the components whose name includes ‘sphere'” (works best if you clear the selection first)
- “Create a new material that is aquamarine” (color materials work well but textured materials don’t as they need access to image files)
- “Add a new layer called ‘Walls'” (because the API uses “layer” instead of “tag” you will need to use the word layer here)
- “Add a new lyaer called ‘Walls'” (it understands typos correctly!)
- “Add a new scene tab with a single line of code” (adding “with a single line of code” often prevents the AI from trying to address all the options with the response, cutting down on tokens)
- “Draw a grid of 100 12 inch boxes, spaced at 24 inches” (correctly draws a 10 x 10 grid of boxes)
- “List the center points of all the objects in the selection” (this will print the coordinates to the Ruby Console using the puts command. Make sure you have the console open before you run this command)
- “List all the unique vertices in the selection” (similar)
- “Randomly erase 50% of the selected objects” (this works, but as mentioned before: use with caution)
- “Open the SketchUp website” (opens the website in a browser, same for the 3D Warehouse)
- “Zoom extents” (you can even control the view)
Generate code for editing
Set the Prompt Prefix to “Use SketchUp Ruby“ (or similar – feel free to experiment) and the Execute Code option to No. Use this to generate Ruby code that you can then copy and paste into your favorite code editor (e.g. my Ruby Code Editor) where you can further edit and execute it. You can use this to create and look up code snippets.
Q&A / Chat
Clear the Prompt Prefix (or set it to “None“ in version 1.0.2) and the Execute Code option to No: Use this to ask the AI anything that does not necessarily generate code (e.g. “Tell me a joke”). You can this way use OpenAI’s regular chatbot behavior.
Screenshots and Videos
Show Off Your Work
Did you create something amazing with this extension? I would love to see it! You can either upload an image in the comment section below (public) or privately send it to me.
Extension Terms and Conditions
BY INSTALLING, ACCESSING OR USING THIS SOFTWARE, YOU (THE USER) AGREES TO BE BOUND BY THE TERMS AND CONDITIONS AS PROVIDED BELOW.
LICENSE:
This is free software, provided under the GPL license (https://www.gnu.org/licenses/gpl-3.0.html).
(c) Alexander Schreyer, alexschreyer.net
DISCLAIMER:
THIS SOFTWARE IS PROVIDED ‘AS IS’ AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHOR OR ANY COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY ARISING FROM, OUT OF OR IN CONNECTION WITH THIS SOFTWARE OR THE USE OR OTHER DEALINGS IN THIS SOFTWARE.
WHERE APPLICABLE, THIRD-PARTY MATERIALS AND THIRD-PARTY PLATFORMS ARE PROVIDED ‘AS IS’ AND THE USER OF THIS SOFTWARE ASSUMES ALL RISK AND LIABILITY REGARDING ANY USE OF (OR RESULTS OBTAINED THROUGH) THIRD-PARTY MATERIALS OR THIRD-PARTY PLATFORMS.
Downloads
You can now find this extension in SketchUcation’s PluginStore and at the download links below. Due to Trimble’s AI-specific rules, this extension cannot be installed from the Extension Warehouse.
ALTERNATIVE DOWNLOADS
Linked below are the installation files (RBZ format) for the current and some previous versions of this extension.
- as_openaiexplorer_2-1.rbz - 2.1
- as_openaiexplorer_2-0.rbz - 2.0
- as_openaiexplorer_1-0-3.rbz - 1.0.3
- as_openaiexplorer_1-0-2.rbz - 1.0.2
- as_openaiexplorer_1-0-1.rbz - 1.0.1
- as_openaiexplorer_1-0.rbz - 1.0
To install these files, follow the SketchUp folks’ instructions. You can also find this extension on GitHub.
Revision History
- Version 2.1 (9/23/2023):
– Better error handling
– Fixed gpt-4 markdown extraction issue
– Added a timer
– Set a 30 seconds read timeout for hung requests - Version 2.0 (7/23/2023):
– Created dialog-based input for more chat-like experience
– Moved error reporting into dialog to reduce pop-ups - Version 1.0.3 (7/19/2023):
– Added license/disclaimer file
– Added better license display on first use
– Added link to OpenAI TOU to menu
– Reverted the prompt prefix to a text box (user can now enter anything)
– Disabled code execution as the default (can be turned on in settings)
– Changed from OpenAI completion model to chat model (OpenAI changes) - Version 1.0.2 (3/28/2023):
– Fixed error handling bug
– Added a double-check for delete requests - Version 1.0.1 (3/24/2023):
– Added cleanup for the returned code/text
– Added error handling for API and a one-time warning for the extension - Version 1.0 (3/18/2023):
– Initial release
Troubleshooting
You can reset this extension by pasting the following code into SketchUp’s Ruby Console (which can be found under the Extensions menu) and executing it (hit Enter):
Sketchup.write_default( "as_openaiexplorer" , "openai_warning" , nil )Sketchup.write_default( "as_openaiexplorer" , "openai_explorer_settings" , nil )Sketchup.write_default( "as_openaiexplorer" , "openai_explorer" , nil )Sketchup.write_default( "as_openaiexplorer" , "disclaimer_acknowledged" , nil )
References
- OpenAI chat completion reference
- OpenAI chat completion API reference
- My initial post about this technology:
Yes, you can use OpenAI’s GPT AI to Create Geometry in SketchUp