Slow performance when converting Text to Numbers

Grasshopper
,
1

Hello,
I have a large amount of Numbers that I get from text document that I load into Grasshopper.
Converting the text to Numbers seems to be rather slow:
Converting 30000 Text objects to numbers takes more than 1 second.

image
image1052×629 51.6 KB

100000 text objects take 3.5s.
These numbers change more than once a second so the conversion is a bottleneck in my definition.
I am also planning on using even more numbers (up to a million) so this is a significant performance issue.
Is there some sort of best practice when converting between text and number types?
I will probably script my own text file parser to extract the numbers directly but I was wondering if there is an easier way without scripting.
Thank you for any help or ideas.

2

Could you share the definition or at least part of it that handles the number creation?

Also, do you have string/text characters like letters in your original text values or just numbers?

Can you not use numbers from the get go, instead of passing text between whatever is generating the numbers and the number result?

3

It will take some fixed time per conversion. I’m not sure how much that is but that’s your lower bounds. What you can try is to remove the overhead imposed by Grasshopper.

The most obvious thing (although unless you’re a programmer not exactly a trivial thing) to try is to read the file using a custom script component and perform the conversion directly. Possibly even using multiple threads to do so.

Two things are important to know:

  1. Should the component update whenever the file changes? I.e. does the script need to be smart enough to re-run itself?
  2. Will there be lines in file which are not valid numbers? I.e. does the script need to handle failures gracefully?
4

I got it down using straight up C#. Nearly 70k numbers in 17 milliseconds.
fast number parser.gh (13.6 KB)

scripted parsing
scripted parsing1465×736 66.2 KB

This will require a Grasshopper which has the new scripting components. I tried using the legacy ones, but they’re crashing now. Here’s the script in case you need it:

  private void RunScript(string File, out object Numbers)
  {
    var lines = System.IO.File.ReadAllLines(File);
    var numbers = new GH_Number[lines.Length];

    for (int i = 0 ; i < lines.Length; i++)
      numbers[i] = new GH_Number(double.Parse(lines[i]));

    Numbers = numbers;
  }

The two main optimisations here are to side-step the line-by-line handling of the data between the first few components, and to directly create GH_Numbers instead of doubles, replacing two rather expensive conversions with a single fast one.

Do note that double.Parse() will throw exceptions if the text isn’t a valid number, so if your file contains empty lines or comments or some other non-numeric data that will have to be handled.

3 Likes