Automated UI Tests with Eto.Forms

[mrhe (Mariusz Hermansdorfer)]

Hey developers,

I’m trying to automate UI integration tests by emulating actual user clicks on UI elements. Why do I need this? Edge cases of type: ‘click on the control, but release mouse button outside of it’, or ‘click on this control, nested inside of another control, nested inside of yet another one’ are difficult co capture with simple unit tests.

It’s relatively straightforward to run these tests with native Eto controls. But Eto.Drawable is not compatible with automation. That’s unfortunate, since all my custom controls are based on Eto.Drawable…

Do you know of any way of making the automation framework aware of the Drawable control?

I’m tagging @curtisw and @fraguada but would appreciate input from anyone knowledgeable in this matter.

My test panel is super simple:

            var mainLayout = new DynamicLayout() { Padding = 10, Spacing = new Size(0, 6) };
            var myDrawable = new Drawable();
            myDrawable.Size = new Size(-1, 100); 
            myDrawable.BackgroundColor = Eto.Drawing.Colors.Red;
            myDrawable.ID = "MyDrawable";

            var myButton = new Button();
            myButton.Text = "MyButton";

            var myPanel = new Panel();
            myPanel.Size = new Size(-1, 100);
            myPanel.BackgroundColor = Eto.Drawing.Colors.Blue;
            mainLayout.AddRow(myButton);
            mainLayout.AddRow(myDrawable);
            mainLayout.AddRow(myPanel);
            mainLayout.AddRow(null);
            Content = mainLayout;

For the tests, I’m using FlaUI and NUnit. The setup is also quite basic:

        [OneTimeSetUp]
        public void OneTimeSetUp()
        {
            _app = Application.Launch(@"C:\Program Files\Rhino 7\System\Rhino.exe");
            Thread.Sleep(15000);

            _mainWindow = Retry.WhileNull(() =>
            {
                var automation = new UIA3Automation();
                return _app.GetMainWindow(automation);
            }, TimeSpan.FromSeconds(60)).Result;
            TestContext.WriteLine($"Main window found: {_mainWindow}");

            _uiPanel = _mainWindow.FindFirstDescendant(cf => cf.ByName("UI_Playground"));
            TestContext.WriteLine($"UI Panel found: {_uiPanel}");
        }

        [Test]
        public void Expander_Exists()
        {
            var descendants = _uiPanel.FindAllDescendants();
            foreach (var element in descendants)
            {
                TestContext.WriteLine($"Name: {element.Name}, AutomationId: {element.AutomationId}, ControlType: {element.ControlType}, Class: {element.ClassName}, Property: {element.BoundingRectangle}");
            }
        }

Which finds the button without issues but completely ignores the Panel and the Drawable:

Name: RhinoReorderTabCtrl_v1:9568, AutomationId: 6625, ControlType: Pane, Class: RhinoTabCtrlWindowClass, Property: {X=1464,Y=226,Width=456,Height=763}
Name: , AutomationId: 7475054, ControlType: Pane, Class: WindowsForms10.Window.8.app.0.20f9772_r42_ad1, Property: {X=1465,Y=248,Width=454,Height=739}
Name: , AutomationId: , ControlType: Pane, Class: Pane, Property: {X=1465,Y=248,Width=454,Height=739}
Name: MyButton, AutomationId: , ControlType: Button, Class: Button, Property: {X=1475,Y=258,Width=434,Height=23}
Name: , AutomationId: , ControlType: Image, Class: Image, Property: {X=0,Y=0,Width=0,Height=0}
Name: MyButton, AutomationId: , ControlType: Text, Class: Text, Property: {X=1666,Y=262,Width=53,Height=15}
Name: MyButton, AutomationId: , ControlType: Text, Class: TextBlock, Property: {X=1666,Y=262,Width=53,Height=15}

Visually, all looks good:

[EDIT]
FlaUInspect’s treeview confirms that the only recognized control is the Button:

[sonderskovmathias (Mathias Sønderskov Schaltz)]

If you ask me they should have made all regular controls with virtual draw methods. So you could have a button and override the stuff you currently make in your drawable.

There are other (minor) caveats with drawable such as not being able to be appointed OkButton property of a dialog.

[mrhe (Mariusz Hermansdorfer)]

I agree that more granular control over the visual appearance of all Eto UI elements would be great. But now that I’ve invested 2 years into creating my own components I’d really like to automate the testing part

[CallumSykes]

Hey @mrhe,

I’m working on this stuff at the moment to make it easier to poke and prod Eto elements. I would say you best resource is actually Eto itself! @curtisw has made a pretty smashing set of tests for Eto here this works great and can let you run and test UI elements. If you’re interested in how I poke UI elements with clicks and keydowns happy to discuss further, very curious how you’re automating too

In the future, hoping to include some of these items inside of the Rhino.Testing nuget to make it easier for y’all.

[mrhe (Mariusz Hermansdorfer)]

Thanks @CallumSykes,

I’d be very much interested in learning how to simulate clicks on various Eto elements, specifically Drawables. I’ve seen the work done by Curtis here:

picoe/Eto/blob/develop/test/Eto.Test/UnitTests/Forms/Controls/DrawableTests.cs

but I’m not sure this approach can automate user interaction.

My current approach is still quite manual. I have a dedicated project with all custom UI controls on a simple panel and go through a set of actions to verify everything works as expected.

Ideally, I’d be able to automate this kind of behavior.

• click on the Expander, assert that it opened and Slider and GradientControl are now visible
• click on the GradientControl in the left corner, assert that its height has changed and it is now editable
• drag one of the GradientStops, assert that the GradientBinding has changed
• click on the GradientControl, release mouse button outside of it, assert that nothing happened

I was hoping to automate these actions with FlaUI but none of my controls are visible to this framework.

[CallumSykes]

Hey @mrhe,

What I created was a helper class that lets me simulate clicks, that is click, drag, key events etc.
It’s not perfect, but it’s working nicely.

I have avoided creating “real” clicks and “real” key events, mostly because if a stray window gets in the way, automated testing planks, or if I move my mouse, it again planks, and also I have to sit and wait whilst my computer runs tests, which is lame, I like to browse the forums when I’m waiting hehe.

Using the below I can simulate a user drawing svg elements, clicking them, dragging them and modifying them etc.
There are some limitations, but mostly it works nicely.

Here is the Perform Click class.

private const BindingFlags Protected = BindingFlags.NonPublic | BindingFlags.Instance;

public static void PerformClick(this Control control, MouseEventArgs e)
{
  void invoker(Control c)
  {
    var onMouseDownMethod = c.GetType().GetMethod("OnMouseDown", Protected);
    onMouseDownMethod.Invoke(c, Protected, null, new object[] { e }, null);
  }

  if (!control.PerformMouseEvent(invoker, e.Location, () => e.Handled, out var handledControl)) return;

  var onMouseUpMethod = handledControl.GetType()?.GetMethod("OnMouseUp", BindingFlags.NonPublic | BindingFlags.Instance);
  onMouseUpMethod?.Invoke(handledControl, Protected, null, new object[] { e }, null);
}

I need to improve it to also perform a “fall through”, which it does not currently do so that if I call it on the Dialog, the click will pass down through the elements, stopping on a e.Handled = true, this does work for my KeyDown simulations though.

/// <summary>
/// Simulates a Key press down and up.
/// Up will not fire unless  down was handled by a control.
/// </summary>
/// <param name="control">The control to send a key event to</param>
/// <param name="keyData">They key to send</param>
/// <returns>Returns False if the key event was not handled by any control, and hence KeyUp was not actioned</returns>
public static KeyResults PerformKeyPress(this Control control, Keys keyData)
{
  if (!PerformKeyDown(control, new KeyEventArgs(keyData, KeyEventType.KeyDown), out var handledControl)) return KeyResults.None;

  bool keyUpHandled = PerformKeyUp(handledControl, new KeyEventArgs(keyData, KeyEventType.KeyUp));
  return new KeyResults(true, keyUpHandled, handledControl);
}

private static bool PerformKeyDown(Control control, KeyEventArgs e, out Control handledControl)
{
  handledControl = null;

  void invoker(Control c)
  {
    var onKeyDownMethod = c.GetType()?.GetMethod("OnKeyDown", Protected);
    onKeyDownMethod?.Invoke(c, Protected, null, new object[] { e }, null);
  }

  control.PerformFallThrough(() => e.Handled, w => w.Children.OfType<Control>().ToList(), invoker, out handledControl);
  return handledControl is not null;
}

private static bool PerformFallThrough(this Control control,
                                        Func<bool> isHandled,
                                        Func<IBindableWidgetContainer, List<Control>> getChildren,
                                        Action<Control> invoker,
                                        out Control handled)
{
  handled = null;

  invoker(control);

  if (isHandled())
  {
    handled = control;
    return true;
  }

  if (control is not IBindableWidgetContainer widget) return false;

  foreach (var childControl in getChildren(widget))
  {
    if (childControl.PerformFallThrough(isHandled, getChildren, invoker, out handled))
      return true;
  }

  return false;
}

Hope that proves useful to you. And to pre-answer I think your next question, yes, I’m hoping to release this at some point (ideally in Rhino.Testing) along with other things to make this easier. But I make no promises as to a date!