Wish: File importer status in %

It would be nice to have a % status when readin in large files. Like now when I am importing a 1.8 GB stl file… how long do I have to stare at that black screen? A minute? 10 minutes? An hour? I have no idea at all…

So what I have done on a file importer I made was to add a % calculation for reading the file and another for processing the data. It’s simple to do and makes a huge difference for anyone in a hurry as black screens with no feedback are frustrating :slight_smile:

(FYI: It was a 38 million polygon scan , and that was the small version… :slight_smile: )

3 Likes

:slight_smile: or I’d like that gone because it slows the import process down. Just display a message when it succeeds or fails.

The time taken to update a status message is negligible, insignificant.

I highly doubt that, at least this is not the case with python scripting. But, still I do not have that deeper understanding in low level programming. Maybe you’re right.

Simple example:

import time

def TST(i=0,max = 10000):
    return "{} out of {}".format(i,max)

if __name__ == "__main__":
    ts = time.time()
    i = 0
    max = 10000
    while i < max:
        # Comment this print for the second test without prompts
        print TST(i,max)
        i+=1
    
    te = time.time()
    print "Elapsed time is {:.2f}".format(te-ts)

TST1 with prompt = 7.30 sec
TST2 wihtout prompt = 0.01 sec

If you ask me that is not negligible, insignifficant.
Add import of 10k or 100k objects to that equation and it goes from 1 min to 20 minutes difference.

Doing that in a tight loop will do that yes, but as a smart person you wouldn’t do that. You’d print once every X (items or time units).

2 Likes

:smiley: and why would you skip objects if printing this is negligible?
You just proved my point

Because if you can handle 100 objects in 0.1 seconds you don’t really have to print 99 to get a sense of progress…

:slight_smile:
Why don’t you admit I’m right about this one?

I have been testing this on a lot lately. In rhino and 3dexperience (the most event driven software I know) the more you prompt, debug and log the slower the software. It turns out that if you completely remove all prompts you get the best performance.

It is not only due to the print command but also the refreshing, redrawing you name it.
By the way that is true also if you do percentages or progress bars.

In Rhino since it all runs as a single process these progress indications are even less useful, because at a certain point Rhino stops responding.

I’ve reported this numerous times on this forum.

I do a lot of status updates - just check the Raytraced HUD…

Yes I know, and if you disable that it’ll be at least 10% faster.
If you stop refreshing and all rendering happens in background even 50+%

Just think about it, if you use Rhino Next Renderer, at a certain point Rhino hangs (stops responding) What it does is it stops refreshing. You probably have more experience rendering but why do you need to see all the pseudo photons on the screen? Just do the rendering with redraw disabled. It will be faster.

Doing nothing will always be fastest. But in the scope of a 1.8GB import the regular status updates are negligible. Comparing with a NOP is useless, @Holo is looking to do something, not nothing.

2 Likes

Yes I understand that. What I wish is to be optional including user to be able to disable that completely. Currently I have to wait 1-10 minutes. All I want is at the end to see a message “DONE” or “FAILED” in less time. Meanwhile I’ll work on something else.

Btw is import linear, can it calculate how much time it needs right after you select a file and it scans how many objects are there?

Seems like whoever developed this didn’t share your thinking :stuck_out_tongue_winking_eye: . It does happen quite fast though.

Update:
One interesting thing:

I left the cursor there so it is visible when Rhino stops responding. It appears that during the actual import, no matter how big the file is Rhino doesn’t stop responding. I recently made some tests and Rhino enters this kind of state only if it is being “told” to wait until the command finishes before executing the next one (like in rs.sleep()). That means during import Rhino wastes even more time, not just for displaying the progress. Unless I’m wrong.

You are so wrong and so right at the same time :smiley:
The example above when import STEPS is a horrible example of printing everyting (Actually it is not printing, but prompting, which is faster, but none the less) and thus slowing down stuff. And as guru @nathanletwory just wrote (alias jesterking) only a silly person would do that (You only do it for bugtracking) as it slows down processes. (I presume your example is a leftover from the 90’s when STEP was new and files were small and cpu’s were slow, so the actual printing represented a minimal load to the system that was already overloaded :smiley: )

The computer knows how many lines of code to import from the file (yes it does, it’s part of the read process) and then you divide that by 100 and then you know when to update it. You can also use time to check, but that I find slower.

Here you can try for your self, this is 1.000.000 operations done three times for you to get the experience. The time difference from the first one is the actual added time, do not compare these as % of the total runtime (I just say that so you don’t start ranting about another thing you don’t know enough about :smiley: )

import rhinoscriptsyntax as rs
import random
import time

Total=1000000
onePercent=Total/100
counter = 0

### Just do the calculus

t1=time.time()

for i in range(Total):
    counter +=1
    rnd = random.random()


print str(time.time()-t1) + " 1. run, no prompting"

### Calculus with statusupdate based on percent

t1=time.time()

for i in range(Total):
    counter +=1
    rnd = random.random()
    if counter %onePercent == 0:
        rs.Prompt( "Processing random value for line "+str(counter)+" of "+str(Total)+" - Random: " +str(rnd))

print str(time.time()-t1) + " 2. run, prompting every percent"


### Calculus with statusupdate based on time

t1=time.time()
t2=time.time()

for i in range(Total):
    counter +=1
    rnd = random.random()
    tmpTime= time.time()
    if tmpTime-t2> 0.25:
        t2=tmpTime
        rs.Prompt ( "Processing random value for line "+str(counter)+" of "+str(Total)+" - Random: " +str(rnd))

print str(time.time()-t1) + " 3. run, prompting every 0.25 second"

Edit:
This is the output of the updated version:

0.995429992676 1. run, no prompting
1.86982727051 2. run, prompting every percent
2.19875335693 3. run, prompting every 0.25 second

So as you see you add 1 second to the processing of 1.000.000 items, so if you divide 1 second with 1.000.000 then you get the actual processing added to each process, not much in other words.

1 Like

Probably, from before when I started. I reported this as https://mcneel.myjetbrains.com/youtrack/issue/RH-51851

As for the OP request, I see we have an ancient report: https://mcneel.myjetbrains.com/youtrack/issue/RH-2164 . I’ll link this topic to it so it gets bumped.

3 Likes

@Holo,

thanks for this code, it does help me to understand something.

I changed the code a bit to check algorithms with different complexities.
There are some weird results on low number of iterations:

import rhinoscriptsyntax as rs
import random
import time
import math

def func1():
    return random.random()

def func2():
    return math.factorial(100)

def func3(n=10):
    if n <= 1:
        return n
    return func3(n-1) + func3(n-2)

Total=1000000
onePercent=Total/100
counter = 0

### Just do the calculus

t1=time.time()

for i in range(Total):
    counter +=1
    #************************#
    #value = func1()
    #value = func2()
    value = func3()
    #************************#


print str(time.time()-t1) + " 1. run, no prompting"
tst1 = time.time()-t1

### Calculus with statusupdate based on percent

t1=time.time()

for i in range(Total):
    counter +=1
    #************************#
    #value = func1()
    #value = func2()
    value = func3()
    #************************#
    if counter % onePercent == 0:
        rs.Prompt( "Processing random value for line "+str(counter)+" of "+str(Total)+" - Value: " +str(value))

print str(time.time()-t1) + " 2. run, prompting every percent"
tst2 = time.time()-t1

### Calculus with statusupdate based on time

t1=time.time()
t2=time.time()

for i in range(Total):
    counter +=1
    #************************#
    #value = func1()
    #value = func2()
    value = func3()
    #************************#
    tmpTime= time.time()
    if tmpTime-t2> 0.25:
        t2=tmpTime
        rs.Prompt ( "Processing random value for line "+str(counter)+" of "+str(Total)+" - Value: " +str(value))

print str(time.time()-t1) + " 3. run, prompting every 0.25 second"
tst3 = time.time()-t1


dif1 = tst2 - tst1
dif2 = tst3 - tst1

print "difference 1: {} sec, difference 2: {} sec".format(dif1,dif2)


Results:

1000 iterations
Algorithm 1
0.00499725341797 1. run, no prompting
0.137016296387 2. run, prompting every percent
0.00799560546875 3. run, prompting every 0.25 second
difference 1: 0.122016906738 sec, difference 2: -0.00700378417969 sec
Algorithm 2
0.0290069580078 1. run, no prompting
0.80908203125 2. run, prompting every percent
0.0179977416992 3. run, prompting every 0.25 second
difference 1: 0.779075622559 sec, difference 2: -0.0120086669922 sec
Algorithm 3
0.0489959716797 1. run, no prompting
0.157005310059 2. run, prompting every percent
0.0439910888672 3. run, prompting every 0.25 second
difference 1: 0.0980224609375 sec, difference 2: -0.0139846801758 sec

100000 iterations:
Algorithm 1:
0.115020751953 1. run, no prompting
0.325035095215 2. run, prompting every percent
0.245025634766 3. run, prompting every 0.25 second
difference 1: 0.210021972656 sec, difference 2: 0.130012512207 sec
Algorithm 2:
1.58716583252 1. run, no prompting
2.43523406982 2. run, prompting every percent
1.73817443848 3. run, prompting every 0.25 second
difference 1: 0.848068237305 sec, difference 2: 0.152008056641 sec
Algorithm 3:
4.56545257568 1. run, no prompting
4.73948669434 2. run, prompting every percent
4.65445709229 3. run, prompting every 0.25 second
difference 1: 0.174034118652 sec, difference 2: 0.0890045166016 sec

1000000 iterations
Algorithm 1:
1.04610443115 1. run, no prompting
1.68716430664 2. run, prompting every percent
2.24222564697 3. run, prompting every 0.25 second
difference 1: 0.641059875488 sec, difference 2: 1.19612121582 sec
Algorithm 2:
15.4585494995 1. run, no prompting
16.9316940308 2. run, prompting every percent
17.3017349243 3. run, prompting every 0.25 second
difference 1: 1.47314453125 sec, difference 2: 1.84318542480 sec
Algorithm 3:
38.4748458862 1. run, no prompting
39.2499237061 2. run, prompting every percent
39.9949951172 3. run, prompting every 0.25 second
difference 1: 0.77507019043 sec, difference 2: 1.52114868164 sec

There is nothing weird about the results with fewer iterations. Fewer just happens to work within the 0.25s, so almost nothing happens - only the check for the timer really. But no printing. Whereas with the one percent you have 100 prints additionally to the checks.

In the higher iteration results most of the additional time probably goes into the checks, not so much the actual printing.

2 Likes

I see, thanks.

My pleasure, that’s how I learned 99% of the little I know too :slight_smile:

And you know something I don’t know, what does this do?:

def func3(n=10):
if n <= 1:
return n
return func3(n-1) + func3(n-2)

the (n=10) part is what intrigues me.