Error in Rhino 8 SR31 when using pandas: unknown locale: en-US [7:1]

Grasshopper
,
1

In Grasshopper when using pandas in a Python3 component, I get an error “1. Error executing script: unknown locale: en-US [7:1]” and have confirmed same behaviour from clients.

The fix:
1: Open Rhino
2: Run ‘ScriptEditor’ command
3: Make new Python Script.
4: Run this code:

import locale
locale.setlocale(locale.LC_ALL, ‘en_US’)
import pandas as pd

This is only after SR31

2

I believe i’m running into the same issue on my client’s PC after update.

I was getting this error in the GH python component:

1. Error running script: partially initialized module ‘pandas’ has no attribute ‘_pandas_datetime_CAPI’ (most likely due to a circular import) [3:1]

After re-installing pandas and numpy libraries through the Python script editor in Rhino is was getting the locale issue. After resolving this with a snippet from another post i am getting the initial Error again.

My client is on Windows btw

4

This is the snippet and possible solution i found but i see you already have it in your code

6

@eirannejad

Could you have a look at this please?

7

Updated from 8.31 to 8.32 but the problem persists

8

@Calcman This is a known issue with pandas on Windows. The underlying problem is that Windows locale names are not very standard and python locale module sometimes fails at processing them.

Run this script and tell me what it prints:

import locale
print(locale.getlocale())

This is what it prints on my machine ('English_United States', '1252')

Also maybe try this other snippet that adds .UTF-8 to the locale. See if this works.

import locale
locale.setlocale(locale.LC_ALL, 'en_US.UTF-8')
import pandas
9

Hi Ehsan, thank you for your help. I’m getting the following from the script environment:

Traceback (most recent call last):
File “file:///C:/Users/Asus/.rhinocode/stage/5uq5u5yf.c5y”, line 2, in
NameError: name ‘locale’ is not defined

The other snippet you shared was already part of my script. It has worked well for about two years. The current situation happened only after update to v8.31

My code:

# r: gsheets, pygsheets, pandas

#https://erikrood.com/Posts/py_gsheets.html
import pygsheets

import os
path = os.path.dirname(ghdoc.Path)

#https://discourse.mcneel.com/t/issues-installing-pandas-on-script-editor/166071/10
import locale
locale.setlocale(locale.LC_ALL, 'en_US.UTF-8')
import pandas as pd
import scriptcontext as sc
from io import StringIO

#authorization
gc = sc.sticky['authorization']

naam = naam

index = 5

# Define a function to get CSV from a specific sheet tab
def get_csv_from_tab(sh, index):
    wks = sh[index]
    data = pd.DataFrame(wks.get_all_records())
    #csv_buffer = StringIO()
    return data.to_csv(sep=';', index=False)
    #return csv_buffer.getvalue()

# Open the Google spreadsheet
sh = gc.open(naam)

# Manually select the tabs by index and store their CSVs in separate variables
then here are a few lines of get_csv_from_tab
10

I was able to resolve this with:

rhino_startup.py

import locale

try:
locale.setlocale(locale.LC_ALL, ‘’)
except:
pass

try:
current = locale.setlocale(locale.LC_ALL, None)

if '-' in current:
    locale.setlocale(
        locale.LC_ALL,
        current.replace('-', '_')
    )

except:
pass
11

Ya I had a script like this by default to fix Windows-style locale names but that ended up messing up other locale based stuff. I am glad this works for you. Thanks for the update.

13

Hi Ehsan, how can we get a permanent fix for this. It means pandas is not robust to include in any tools in Rhino?

14

I needed to reset Python Runtime in Rhino for this to work. Very fragile fix

15

Let me take another look and see if I can come up with a solution.
RH-81130 ScriptEditor: Python 3 to set locale before running script

16

Hi @eirannejad, I’m getting quite desperate for a fix here. Any progress? :grimacing:

17

Hello Ehsan,

I would like to report that the issue still persists in 8.33. The behavior we are witnessing I describe below.

When there is a component on the canvas that is referencing pandas like this

#r: pandas

import pandas

it still throws with error message “Error running script: unknown locale: en-US [2:1]” at first, and in subsequent components with pandas import or after component reload/re-enabling it throws “Error executing script: partially initialized module ‘pandas’ has no attribute ‘_pandas_datetime_CAPI’ (most likely due to a circular import)”.

We had to create our own custom excel reader because of serious reasons (readers available in public plugins weren’t allowing us to partially read huge tables and therefore save loading time) and now our custom solution is basically invalidated, putting us in trouble.

Having this issue solved would help us tremendously

Thank you

EDIT:

I tried to create minimal example, sample component is left hanging on error “partially initialized module…” no matter what I do. Further details below

image
image1050×675 77.9 KB

Traceback (most recent call last):
File “rhinocode:///grasshopper/1/2cbced09-bbc9-4430-8671-4e1913092f66/8cc82c59-f560-4590-a760-9314106a9910”, line 3, in
File “C:\Users\AntPas.rhinocode\py39-rh8\site-envs\default-LyIk9k8B\pandas_init_.py”, line 80, in
from pandas.core.api import (
File “C:\Users\AntPas.rhinocode\py39-rh8\site-envs\default-LyIk9k8B\pandas\core\api.py”, line 1, in
from pandas.libs import (
File "C:\Users\AntPas.rhinocode\py39-rh8\site-envs\default-LyIk9k8B\pandas_libs_init.py", line 18, in
from pandas._libs.interval import Interval
File “pandas/_libs/interval.pyx”, line 1, in init pandas._libs.interval
File “pandas/_libs/hashtable.pyx”, line 1, in init pandas._libs.hashtable
File “pandas/_libs/missing.pyx”, line 40, in init pandas._libs.missing
AttributeError: partially initialized module ‘pandas’ has no attribute ‘_pandas_datetime_CAPI’ (most likely due to a circular import)

EDIT II:

The last SR we have deployed where things seem to be working correctly is SR30. Colleague tested the minimal example above and it references pandas without issues. Unfortunately we haven’t deployed SR31/32, so I can’t pinpoint exactly when that got broken.

18

@Timo_Harboe_Nielsen @Ukrutony

Rhino 8 has not received any changes related to handling of locales in its python environment so if Rhino 8 > SR30 is behaving differently, I am almost entirely sure this is not the scripting environment.

This is also a complicated issue where I do not control all the ends. Here are a few things to note:

  • The problem is Windows reporting BCP-47 locale names (en-US with a dash) instead of POSIX-style names (en_US with an underscore). This throws off python’s locale module on Windows and affects any python module importing the locale. pandas just happens to be the most popular one here :smiley:
  • Python 3.12+ have reworked the locale module implementation on Windows to fix this problem. So I expect we should not be seeing this in Rhino 9 that uses python 3.13
  • We shipped Rhino 8 with a system that script editor would attempt to fix the locale name on startup but I quickly realized this affects Rhino’s localization formatting since the locale is an application-wide setting.
  • So far a variation of this workaround has worked for most people.

With the help of Claude (to get more historical context on this error), I made a python module that attempts for fix this in two ways: Correct the locale name from BCP-47 to POSIX, and also override the locale.getlocale function so it does not throw exceptions.

Put this somewhere on your python search paths and import before pandas like this:

import fix_locale
import pandas

It can also be included as a Rhino startup command like:

_-ScriptEditor _R (
    #! python 3
    import fix_locale
)

Let me know if this works. If the patching of locale.getlocale resolves the problem, I can put that in Rhino permanently.

fix-locale.py (2.4 KB)

20

Hello Ehsan,

Thank you for your response. I feared that the locale issues are something out of your direct control. What caused the change in behavior between the Rhino versions I can’t tell, but it really is present. Anyway:

I confirm that the fix_locale script that you provided fixes the issue, when used before we attempt to reference pandas assembly (or any other, as you explained above)

The issue we have now is that we don’t want to change python settings of numerous users’ computers (linking each one to the fix_locale.py so that we can import it). Therefore I stole the code from your .py and placed in into Py3 directly and it seems to work as well. Does that mean we can include it into our “faulty” components and thereby get immediate patch? (until global solution is officially released)

However, we would still prefer “official” patch, because it’s unclear what components we made would break with 8.33 because of this issue. There’s risk that we would have to patch multiple scripts.

21

Ya that makes sense and thanks for confirming the fix. Let me put the workaround that overrides locale.getlocale into then next Rhino 8 build and I will send you a link to test. If that works it means I don’t have to deal with ‘correcting’ the locale and potentially changing how Rhino localization behaves.

Will update you with a link soon

22

Hi Ehsan.
I just tested becuse Antonin is on vacation.
I have testen on:
8.33.26188.13001
8.34.26215.11001
8.35.26212.10001
Only the 8.35 you send it is working again.
Do you need me to test more?

23

Nope. It if works on 8.35 is it there from now on and is carried to Rhino BETA as well. Thank you so much for being patient and testing this.

24

Thanks for fixing it ! ! !