Grouping, Sorting, and Filtering

Edited: 9/13/2023

The Problem with Filtering

In the first post of this thread, I laid out some techniques for how to filter content based on a data type’s attribute or user text value. It worked, but with one major flaw.

The Filter Content component had an input, called Key, which let you define what attribute or user text key that you wanted to use during the filtering operation. Because this was set on the Filter Content component itself, it meant that you could only ever use a single key during the filtering operation.

So, if you wanted to filter the incoming content for floors whose (Area > 2,250) and (Area < 2,500) sf, for example, you’d be fine. But, if you wanted to create more complex rules where you wanted to filter for (Area > 2,250) and (Area < 2,500) and (Use Type = Office) then you’d have to use two separate Filtering Content components because you’re using two unique “Keys” (ie. Area and Use Type).

You can probably imagine that this would soon become untenable as filtering conditions become more elaborate or bespoke.

Complex Filtering

The solution to the problem above is to remove the “Key” input from the Filtering Content component and move it upstream into each of the various Filtering Rule components. In this way, you can define both the “Key” and the “Condition” that you want to filter for in the same component. You can then combine those various filters together using any of the logical operators (ie. AND, OR, NOT, INTERSECTION, and UNION) to form complex filtering rules.

For example, let’s say we wanted to filter all of the floor plates in the original example for the following condition: (Area > 2,250) and (Area < 2,500) and (Use Type = Office)

First, we would create a new filtering rule using the Greater Than Filter. We can right-click on the “Key” input and select User Text from the drop-down combo box and then Area from the tree view control. Lastly, we can create a numeric slider (set to 2,250) and connect that to the value. This filter (when connected to the Filter Content component) will then check all incoming objects to see if they have a User Text key called “Area”. If it does, it retrieves that value and checks it against the defined condition (ie. Area > 2,250).

Now, if you used the any of the Filtering components prior to today, you may have noticed a subtle change other than the placement of the “Key” input. Prior to today, you used to have to right-click on the component and specify whether you wanted to use By User Text Key or By Attribute. This selection would prefilter the set of values that were visible in the “Key” input.

Now, you no longer need to do this. Instead, you simply need to select the appropriate value from the drop-down list that appears in the combo box when you right-click on the “Key” input.

If for example, you wanted to filter objects by their Layer Name, you would first select the Model Object item in the combo box. Then in the tree view control, you would expand the Layer root node and finally select the Name child node.

Putting it Together

Now that we know how to create the filtering conditions, we can combine them using any of the logical operators (if you want to filter objects using more than one rule). In the example above, we wanted to check for the following: (Area > 2,250) and (Area < 2,500) and (Use Type = Office). You’ll notice that each of the conditions (the stuff in parenthesis) are combined using the keyword and. This means that all three conditions need to be true in order for the content to be passed through the Filtering Content component.

There is a component called Logical AND Filter which combines two rules together… however, we have three conditions. To combine more than two rules together using a logical AND condition, we can use the Filter Intersection component which returns true only if all input filters evaluate to true.

Once we have created the Intersection filter, we can pass that to the Filter Content component and it will evaluate all three of the defined conditions and filter the content accordingly. The final definition should look something like this.

We realize that this is not an insignificant change and that you may need to rework existing definitions in order to accommodate this. However, we do think this is a much more powerful solution for being able to create complex filtering conditions in order to manage your data effectively.

As always, we rely on your feedback. Please let us know if you have any questions or concerns about how to effectively filter content in your definitions.

3 Likes

@AndyPayne Personally I think this is great. And I like fact that I can feed string values into the filter components as this could enable workflows such as allowing users to enter their own filter criteria easily without having to get into GH as we could read their filter criteria from excel or even prompting.

For instance “show me all floors of type office with area greater than 2500 sq ft but no greater than 3600 sq ft”

I could query a string like this and extract “office”, “area”, “>2500”, <“3600” and have those be the inputs for a dynamic filtering operation.

I guess something like this would be better suited to scripting via filtering operation API access but either way I see this having the potential to be very powerful.

So that’s my 2 cents, I like the change. Curious on others thoughts…

1 Like

It took a moment to understand the changes but I think it’s a good solution.

The only thing I would currently change is the naming on the Unit System and User Text components.

Looking at other components I don’t think the U-… abreviation is the consistent way of naming.

1 Like

Hi @Intuos,

Now you have the two ways to invert any Filter one more implicit and another more explicit.

1 Like

@AndyPayne On the surface, these look like very good changes to filtering! Fully agree with the choices you’ve made. Will most likely already test drive them in the next couple of days.
I like how the filtering is now no longer taking place inside the Filter or Group Content components. Now that it is moved to the rules, I think it makes reading definitions a lot easier (inputs are all in one place on the left).

One of the things I don’t really like is the way you had to pick between User Text or Attributes using a dropdown. From what I read the only change was where you have to right click? I always had to set the components to User Text, shouldn’t that be the default mode? I would think its the most common of the two to use.
Why is there even a dropdown for User Text or Attributes? Is the expectation that more things will be added to expand the list? Otherwise a simple checkbox would save us a click every time the input has to change from one to the other :wink:

@kike thanks for adding the invert option for filters, looks really convenient to have it work both ways! I presume you can also invert the Filter component output then? Sorry, I should really check before asking. :upside_down_face:

Thanks again to both of you for adding these really nice features!

Hi, Andy Payne

What is this plugin in the picture,“A plugin called (Rhino)”,Where can I find it?

It’s not a plug-in. It’s a set of native components (available in Grasshopper for Rhino 8) which deal with Rhino objects/data types

I see. Thank you!

I’m having trouble filter by a user text attribute that is not assigned to all objects. I would like to find all geometry that has the user attribute “estimated” when I use the text pattern or text contains widgets it gives me an error and won’t return any results through there is geometry that contains the key estimated. I tried the wildcard and the t contains widgets.

Is this a bug or is there another workflow that would correct this, for example a way to filter for geometry with the key “x” and then to check if “x” met a certain condition.

Figured out a work around which is to filter out all the model objects that contain the specified key prior to filtering by an attribute of that key. It would be nice if filtering by an attribute of a key returned results even if some of the objects don’t have that key. They could return a null value if the key was not found.

For anyone encountering the same issue, see my grasshopper definition below:

Can you post your Rhino file and Grasshopper file so I can take a look. It can be a simplified example if you want as long as it’s showing the error you’re seeing.

See attached.

Another issue I noticed would be that you need to load the entire model into memory and then filter. If you are familiar with web design this would be like loading your entire database into the browser and then filtering on the front end. Good luck not crashing your browser!

Most databases have a way to filter data on the backend, so you would define what it is you want to get within your API call so you are only getting what relevant data on the front end.

One way to do this would be to have filters attached to the Query component. There are some there already but not a general filter component. Right now its pretty janky when using this query component on a large model.

filter by key.gh (13.3 KB)

Yes this has been requested before. It is on our todo list.

You didn’t attach your Rhino file so the Grasshopper definition you sent didn’t really help much. Nevertheless, I tried to repeat the conditions you setup. I created 8 curves in my Rhino document and assigned a User Text key/value pair on 4 of those curves (so some had the user text while others did not). The User Text Key was estimateId and the value I assigned a number of 100 but it could be anything really.

Now, in Grasshopper, if I feed the contents of the Query Model Objects component into a User Text component, I can see from the Keys output that some have the estimateId key while others do not. So, we need to create a filter to check the value of this key.

Here, I’ll use the Match Text Filter component. I right-click on the Key input and select User Text from the drop-down menu and then select estimateId as the Key I want to check against. Now, when the filter component goes through all of the objects, it will look to see if the object contains a key called estimateId and if it does, it will try to see if the “pattern” that we supply matches the value of that key (ie. 100). Now, I could specifically try to only return objects whose key matches estimateId and whose value is 100… but it seems like from your question that you don’t really care about the value… just whether the key is there. So, we can simply pass a wildcard character (ie. *) into the Pattern input and this will create a filter where if there is anything at all in the value slot under the estimateId key, then it will pass as True. And that’s it.

If you look at the definition below, you can see that I’ve passed in 8 objects, but only 4 are returned on the output of the Filter Content component (all of which contain the key estimateId). There is a warning on the Match Text Filter component telling me that 4 of the objects did not contain the key that I specified (ie. estimateId) but that’s ok here since I know some of them do and some of them don’t. This warning can be ignored.

FilterByUserText.3dm (68.1 KB)
FilterByUserText.gh (12.9 KB)

Thanks for posting. Will take a look. This is exactly what I was trying to do but for some reason, the match text with the wildcard wasn’t returning any objects. I thought that if not all the objects had the specified key then the filter didn’t work. Will compare this to what I set up and let you know if there is a bug or it was simply a user error.