UI Automation with MSAA

  • Thread starter Thread starter Steve
  • Start date Start date
S

Steve

I hope this is an appropriate group to post this question. If it is
not I would appreciate someone pointing out a more appriate group.

I hope someone can help me with this as I have been fighting this
issue for several days now.

I am writing an app which will automate some tedious manual work have
to do in my job. The way the solution will work is to basically
execute another application (third party commercial software package)
then perform actions (ie button pushes, menu item selections etc.) on
the UI elements of the newly started application. I have this
solution working for the most part utilizing MSAA to get references to
and manipulate the various UI elements of the applcation. However I
have one problem I have not been able to solve. This problem involves
what looks like a listview or grid control (although the class name
returned by Spy++ is WindowsForms10.Window.8.app3). What I am trying
to do is to step through the items (rows) in this grid/listview,
selecting the row, reading the text out of the first column then click
a button on the same parent as the grid/listview. Doing this sequence
of actions should cause the other app to open the selected document in
it's main window. As I said this is working for the most part. The
only problem I am having is the select of the item (row) which means
when I "click" the button I am always open the first document
(selected by default when the window opens) in the list. I can
correctly get the collection IAccessible objects referencing the rows
of the grid/listview and loop through that collection returning the
text in the first column of each. However when I attempt to force a
row to be selected...nothing hapens.

Ok there is what I am trying to do and the problem I am having now
here is the code I am using.


// Get the list of row items from the project list
grid
Rows = ProjectList.GetChildren
(AccessibleUIItemType.Row);

// Get the number of items in the list
numProjects = Rows.Count();

// Get the Row item associated with curProject
MSAAUIItem Row = Rows[curProject];

// Make sure this is the selected row
Row.Select(SelFlags.TAKEFOCUS |
SelFlags.TAKESELECTION);
// The statement above seems to have no effect

// Get the project name. This will be the name of the
rows first child
ProjectName = Row.Properties.Name;

All of the above statements except the Row.Select statement do excatly
what I would expect.

Can anyone shed some light on what I might be doing wrong here.

I also have a somewhat different (but related) question, why are none
of the windows in this application I am trying to automate "normal"
windows? They all seem to have this wierd "WindowsForms10.Window.
8.app3" (or something similar) class where I would have expected
"SysListView32".

Thanks in advance for any help
Steve
 
Hi Steve,
just a wild hunch (but it reminded me of a problem I had with a grid
on a tabbed notebook): Are you sure (I mean _really_ sure) that the
grid has the focus?
In my case the tab page was selected, but the focus was not in the
grid - and i could set Row.Select(...) all day long without any
effect.
Try explicitly giving the grid the focus, and then select the wanted
row.
// Make sure this is the selected row
Row.Select(SelFlags.TAKEFOCUS |
SelFlags.TAKESELECTION);
// The statement above seems to have no effect
Click to expand...

HTH
Helmut Giese
 
Hi Steve,
just a wild hunch (but it reminded me of a problem I had with a grid
on a tabbed notebook): Are you sure (I mean _really_ sure) that the
grid has the focus?
In my case the tab page was selected, but the focus was not in the
grid - and i could set Row.Select(...) all day long without any
effect.
Try explicitly giving the grid the focus, and then select the wanted
row.


HTH
Helmut Giese
Click to expand...

Helmut,

Thanks for the suggestion...I will try it. If it does work I will be
somewhat shocked as I would have expected the code as written to force
focus to the actual row of the grid. It is my understanding from the
docs on MSAA that the TAKEFOCUS flag does exactly this.

Anyway I will try first giving the grid the focus then calling my row
specific code...I'll let you know how it works out.

Thanks,
Steve
 
No success.

Here is what I have tried.
1) Set focus and select the IAccessible "Row" object by calling
Row.Select(SelFlags.TAKEFOCUS | SelFlags.TAKESELECTION)
as mentioned before, this does not work.

2) First setting focus to the rows parent grid window then setting
focus and selecting the Row like this.
ProjectList.Select(SelFlags.TAKEFOCUS);
Row.Select(SelFlags.TAKEFOCUS | SelFlags.TAKESELECTION)
This also did not work.

3) First setting focus to the grids parent window then setting
focus to the grid then setting focus and selecting the row
OpenProjectUI.Select(SelFlags.TAKEFOCUS);
ProjectList.Select(SelFlags.TAKEFOCUS);
Row.Select(SelFlags.TAKEFOCUS | SelFlags.TAKESELECTION)
This also did not work.

I am nearly positive that the focus is changing as expected but the
selection never does. I say that the focus part is working because if
I place a breakpoint in my code to pause execution (after setting
focus) and press the down arrow key, the keypress go to the grid and
the selected row changes.

In order for my automation of this application to work, I must be able
to switch the selection from one item in the list to the next because
the next action is to be performed on the selected item.

Any other thoughts?

Thanks,
Steve
 
Ok I spoke to soon in my last post. It seems that the third try
(described in my previous post) does in fact work. However the
highlight is not being moved to the newly selected row. The only way
I can tell that the selected row has changed is that the little
pointer icon (which indicates current row) is moved to the row of my
choosing.

Now although the code mentioned does seem to work in that it does move
the current row icon (though not the row selection hightlight) the app
being automated does not seem to regocnize that the current row has
changed. IOW words after changing the current row (as per the code in
try three of my previous post) then performing another UI action which
is to operate on the currently selected item in the list...the action
is performed againt the highlighted row not the one I selected and
which has the current row indicator.

Any ideas?

Thanks,
Steve
 
Hi Steve,
whow, progress - at least a bit.
Ok I spoke to soon in my last post. It seems that the third try
(described in my previous post) does in fact work. However the
highlight is not being moved to the newly selected row. The only way
I can tell that the selected row has changed is that the little
pointer icon (which indicates current row) is moved to the row of my
choosing.

Now although the code mentioned does seem to work in that it does move
the current row icon (though not the row selection hightlight) the app
being automated does not seem to regocnize that the current row has
changed. IOW words after changing the current row (as per the code in
try three of my previous post) then performing another UI action which
is to operate on the currently selected item in the list...the action
is performed againt the highlighted row not the one I selected and
which has the current row indicator.

Any ideas?
Click to expand...
sorry <throwing hands up in despair> I'm a newbie in C# - your
description just rang a bell.
Besides: no matter what language, I am definitively _not_ a GUI type -
I am always fighting them.
Let's hope someone else with "real knowledge" chimes in.

This being said another faint memory comes to mind: I seem to remember
that being _selected_ and being the _current_ element ('element' being
a row in your case) need not always be identical. However, this
definitively concerns a different GUI toolkit - but then, in a way
they all are comparable. Maybe it is worthwhile to check if 'selected'
and 'current' (or 'active' or whatever) are necessarily synonyms.

But in writing this it reminds me of an advice I gave to someone not
too long ago (not C# though): If you can send key strokes to your app
life may be a lot easier:
- Send 'Home' to make sure you're at the 1st position, then
- send as many 'down' keys as needed (just insert a small delay
between key strokes).
BTW this can also make menu traversal a lot easier.

Well, that's all which came to mind - just guessing.

I am off for today. Good luck
Helmut Giese
 
Clear this up for me, you are doing MSAA or MSAA through UIA, sounded like
the latter, wasn't really sure.

Are you sure it is a row? I seem to recall this being a dataitem in UIA

--
Vapordan
Shameless Author Plug
ASP.NET 4 by Example only $20
OWC Blackbook www.lulu.com/owc

Steve said:
I hope this is an appropriate group to post this question. If it is
not I would appreciate someone pointing out a more appriate group.

I hope someone can help me with this as I have been fighting this
issue for several days now.

I am writing an app which will automate some tedious manual work have
to do in my job. The way the solution will work is to basically
execute another application (third party commercial software package)
then perform actions (ie button pushes, menu item selections etc.) on
the UI elements of the newly started application. I have this
solution working for the most part utilizing MSAA to get references to
and manipulate the various UI elements of the applcation. However I
have one problem I have not been able to solve. This problem involves
what looks like a listview or grid control (although the class name
returned by Spy++ is WindowsForms10.Window.8.app3). What I am trying
to do is to step through the items (rows) in this grid/listview,
selecting the row, reading the text out of the first column then click
a button on the same parent as the grid/listview. Doing this sequence
of actions should cause the other app to open the selected document in
it's main window. As I said this is working for the most part. The
only problem I am having is the select of the item (row) which means
when I "click" the button I am always open the first document
(selected by default when the window opens) in the list. I can
correctly get the collection IAccessible objects referencing the rows
of the grid/listview and loop through that collection returning the
text in the first column of each. However when I attempt to force a
row to be selected...nothing hapens.

Ok there is what I am trying to do and the problem I am having now
here is the code I am using.


// Get the list of row items from the project list
grid
Rows = ProjectList.GetChildren
(AccessibleUIItemType.Row);

// Get the number of items in the list
numProjects = Rows.Count();

// Get the Row item associated with curProject
MSAAUIItem Row = Rows[curProject];

// Make sure this is the selected row
Row.Select(SelFlags.TAKEFOCUS |
SelFlags.TAKESELECTION);
// The statement above seems to have no effect

// Get the project name. This will be the name of the
rows first child
ProjectName = Row.Properties.Name;

All of the above statements except the Row.Select statement do excatly
what I would expect.

Can anyone shed some light on what I might be doing wrong here.

I also have a somewhat different (but related) question, why are none
of the windows in this application I am trying to automate "normal"
windows? They all seem to have this wierd "WindowsForms10.Window.
8.app3" (or something similar) class where I would have expected
"SysListView32".

Thanks in advance for any help
Steve
Click to expand...
 
Clear this up for me, you are doing MSAA or MSAA through UIA, sounded like
the latter, wasn't really sure.

Are you sure it is a row? I seem to recall this being a dataitem in UIA

--
Vapordan
Shameless Author Plug
ASP.NET 4 by Example only $20
OWC Blackbookwww.lulu.com/owc


- Show quoted text -
Click to expand...

I am doing plain MSAA (not MSAA via UI). It was neccessary to do this
as a large percentage of the UI elements I needed access to are not
seen by UI Automation (using UISpy).

Yes I am sure it is a row item. This is according to AccExplorer.

Thanks,
Steve
 
Back
Top