4/24/2021 UIASpy - UI Automation Spy Tool - Page 4 - AutoIt Example Scripts - AutoIt Forums

Existing user? Sign In   Sign Up

Browse Wiki AutoIt Resources  FAQ Our Picks Search... 

Forums Downloads Calendar Forum Rules

 Home  AutoIt v3  AutoIt Example Scripts  UIASpy - UI Automation Spy Tool

UIASpy - UI Automation Spy Tool  Sign in to follow this   Fo

By LarsJ, December 1, 2018 in AutoIt Example Scripts


 PREV 1 2 3 4 5 NEXT  Page 4 of 5  

Exit Posted May 13, 2019 (edited)

Exit
At @LarsJ :      You're great! Thank you so much.

Now I have 2 1 questions:

 On 5/13/2019 at 5:08 PM, LarsJ said:

Active Members
 154
The problem is that there are two visitor number controls that you can see when printing the entire tree structure:
1,181 posts

How to print the entire tree structure?

Just found the answer here myself. The search button is your friend. 🙂

 On 5/13/2019 at 5:08 PM, LarsJ said:

And you need the second control to read the text

How could I detect that there are two controls and that I should use the second one without looking at the entire tree structure rst? 
How did you nd out?

Edited May 14, 2019 by Exit

I answered the rst question myself.

App: Au3toCmd              UDF: _SingleScript()                             

Exit Posted May 16, 2019

Exit
Now I've worked a bit in UIASpy and I'm really excited.
@LarsJ  made a great tool. 

Now I have a few comments on the handling of the function keys.

The keys should not be handled as hotkeys but as accelerator keys.
At least if you use common keys like F1-F7.
Active Members
These keys are used in many applications.
 154
For example, the key F1, which is used in SciTe to call syntax help.
1,181 posts
In order to use them, you always have to switch o the 'detect element' function in the UIASpy menu.

Now I have found the solution for me: 

Just use the function keys in shifted mode.

https://www.autoitscript.com/forum/topic/196833-uiaspy-ui-automation-spy-tool/page/4/ 1/8
4/24/2021 UIASpy - UI Automation Spy Tool - Page 4 - AutoIt Example Scripts - AutoIt Forums
Simply add a plus sign (+) at the beginning of the le 'UIASpy_Menu.au3' for the hotkey de nitions.
So the keys F1-F7 are again available for other applications.

Global $aFuncKeys = [ [ "+{F1}", "UIASpy_F1" ], _ ; F-key, Function

[ "+{F2}", "UIASpy_F2" ], _
[ "+{F3}", "UIASpy_F3" ], _
[ "+{F4}", "UIASpy_F4" ], _
[ "+{F7}", "UIASpy_F7" ], _
[ "+{F8}", "UIASpy_F8" ] ]

If someone wants to copy it, just should do it. 

Thanks again to @LarsJ  

App: Au3toCmd              UDF: _SingleScript()                             

philkryder Posted May 20, 2019

Adventurer
got an error when trying to run the UIASpy.au3

P
I just downloaded it today.

line 3787 looks like this:

Local $aElemIndexes = ($aElems[$iIdx][7])[1], $aIndex, $jCnt = 0
Active Members  
0
102 posts here is from the console:

>"C:\Program Files (x86)\AutoIt3\SciTE\..\AutoIt3.exe" "C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.au3" /run /pr

/ErrorStdOut /in "C:\#\__AutoIT\!! try IUI automation - may work on controls without names\UIAExamples\UIASpy.au3" /UserParams    
+>14:03:15 Starting AutoIt3Wrapper v.18.708.1148.0 SciTE v.4.1.0.0   Keyboard:00010409  OS:WIN_10/  CPU:X64 OS:X64  Environment(Lang
 CodePage:0  utf8.auto.check:4
+>         SciTEDir => C:\Program Files (x86)\AutoIt3\SciTE   UserDir => C:\Users\pkryder\AppData\Local\AutoIt v3\SciTE\AutoIt3Wrapper  
SCITE_USERHOME => C:\Users\pkryder\AppData\Local\AutoIt v3\SciTE 
>Running AU3Check (3.3.14.5)  from:C:\Program Files (x86)\AutoIt3  input:C:\#\__AutoIT\!! try IUI automation - may work on controls witho
names\UIAExamples\UIASpy.au3
+>14:03:16 AU3Check ended.rc:0
>Running:(3.3.14.5):C:\Program Files (x86)\AutoIt3\autoit3.exe "C:\#\__AutoIT\!! try IUI automation - may work on controls without
names\UIAExamples\UIASpy.au3"    
--> Press Ctrl+Alt+Break to Restart or Ctrl+Break to Stop
"C:\#\__AutoIT\!! try IUI automation - may work on controls without names\UIAExamples\Includes\UIASpy_ElemInfo.au3" (3787) : ==> Subs
non-accessible variable.:
Local $aElemIndexes = ($aElems[$iIdx][7])[1], $aIndex, $jCnt = 0
Local $aElemIndexes = ($aElems[$iIdx][7])^ ERROR
->14:03:17 AutoIt3.exe ended.rc:1
+>14:03:17 AutoIt3Wrapper Finished.
>Exit code: 1    Time: 2.566
 

LarsJ Posted May 22, 2019

The Cool Code Company
You can download a version of UIASpy that works on all Windows 10 versions in this post.

File Explorer,  ROT objects,  UI Automation,  Windows Message Monitor

Compiled code: Accessing AutoIt variables,  DotNet.au3 UDF,  Using C# and VB code
MVPs Shell menus: The Context menu,  The Favorites menu.  Shell related: Control Panel,  System Image Lists
Graphics related: Rubik's Cube,  OpenGL without external libraries,  Navigating in an image,  Non-rectangular selections
 1,118
Control related: Hot-Track Enabled Toolbar Menu,  Simulating a modeless Choose Color Dialog,  Colors and fonts in TreeViews
1,822 posts
ListView controls: Colors and fonts,  Multi-line header,  Multi-line items,  Checkboxes and icons,  Incremental search
ListView controls: Virtual ListViews,  Editing cells,  Data display functions

cyxstudio Posted September 18, 2019


Seeker

https://www.autoitscript.com/forum/topic/196833-uiaspy-ui-automation-spy-tool/page/4/ 2/8
4/24/2021 UIASpy - UI Automation Spy Tool - Page 4 - AutoIt Example Scripts - AutoIt Forums
i just went through the codes and i realise that you did not use third party control functions like controlgettext(). 

C May i know how do you get properties of child control?

Active Members I tried it and it was able to detect controls that autoit info is not able to such as power builder data window.
0
23 posts

junkew Posted September 18, 2019

Universalist
See faq31 that leads to iuiautomation thread in examples section many examples are given to retrieve properties

FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets

MVPs

 407
2,648 posts

supersonic Posted October 22, 2019 (edited)

Universalist
LarsJ, another great tool of yours!

S
Here in our company we are using a terminal program called "Mocha TN5250 for Windows 7/8/10" from https://mochasoft.dk/ . Currently
terminal text using ControlSend() - which works 95%. It is somehow "sensible"; e. g. it won't work if - for whatever reason - window lost foc

I'm not able grab any information with AutoIt Window Info Tool and even your tool can't see this control/part of the GUI.
Active Members
4 Do you have ever heard about "invisible" controls? And do you have a clue how to address such controls?
598 posts
Edited October 22, 2019 by supersonic

junkew Posted October 22, 2019

Universalist
If you are in control of the installation check if the activex control of mocha is there then you could use objcreate function. If not search fo
terminal dll. If you have no development background easiest will be screenscraping with controlsend and winactivate commands. Its hard
terminal emulators as there are many around. 

https://www.ibm.com/support/knowledgecenter/en/SSEQ5Y_6.0.0/com.ibm.pcomm.doc/books/html/emulator_programming07.htm

MVPs  

 407
2,648 posts

FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets

supersonic Posted October 23, 2019 (edited)

Universalist
junkew, thank you.

S
Yes, I am in control of the installation/system but Mocha ActiveX is not (and will not be) available, unfortunately. The ActiveX control is also
grab text from running sessions initiated by the standard Mocha client. And as far as I understand EHLLAPI it also isn't able to grab session
can build my own terminal application... Well, that's somehow beyond the goal 😊

Active Members So, I will stick with Win*() and Control*() functions - that's good enough for now...
4
598 posts But... If anyone else have a clue why there could be a hidden/invisible control and how to grab text from it... I would be very pleased...
 

Edited October 23, 2019 by supersonic


LarsJ Posted October 23, 2019

https://www.autoitscript.com/forum/topic/196833-uiaspy-ui-automation-spy-tool/page/4/ 3/8
4/24/2021 UIASpy - UI Automation Spy Tool - Page 4 - AutoIt Example Scripts - AutoIt Forums
The Cool Code Company
I think these terminals/consoles are simply windows and not controls. The text in the windows is written through the window's device con
with GDI functions like DrawText. That's why they can't be automated. There are a few posts on the same topic here.

File Explorer,  ROT objects,  UI Automation,  Windows Message Monitor

MVPs
Compiled code: Accessing AutoIt variables,  DotNet.au3 UDF,  Using C# and VB code
Shell menus: The Context menu,  The Favorites menu.  Shell related: Control Panel,  System Image Lists
 1,118
Graphics related: Rubik's Cube,  OpenGL without external libraries,  Navigating in an image,  Non-rectangular selections
1,822 posts
Control related: Hot-Track Enabled Toolbar Menu,  Simulating a modeless Choose Color Dialog,  Colors and fonts in TreeViews
ListView controls: Colors and fonts,  Multi-line header,  Multi-line items,  Checkboxes and icons,  Incremental search
ListView controls: Virtual ListViews,  Editing cells,  Data display functions

junkew Posted October 23, 2019

Universalist
Building your own terminal emulator is not that hard just long strings send back and forth to a mainframe with some bytes representing c
elds so it depends on your goals.

Hooking drawtext is an option but hard to interpret and control.

I have never seen a terminal emulator that has no option to grab to text on clipboard. Whats the macro language of mocha o ering?
MVPs
I remember some open source terminal emulator is around on github but do not remember details.
 407
2,648 posts

FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets

junkew Posted October 23, 2019

Universalist
https://www.codeproject.com/Articles/9615/Using-EHLLAPI-in-C

FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets

MVPs

 407
2,648 posts

supersonic Posted October 24, 2019

Universalist
@LarsJ - Sorry for bothering you again: Maybe it is a GDI window (I don't know how to evaluate that) but it is funny that rows/cols can be s
mouse (or all text by Ctrl+A). Is this also typical for GDI windows?

S @junkew - Thank you for motivating me to code my own terminal emulation. 😊 Here in our company Mocha terminal _must_ be used and
unfortunately there is no way around it. I can live with that. Of course, Mocha can grab text (by keyboard, mouse and menue, macro [= di
all users]). My aim was to automate text grabbing (and any further processing) by a litte AutoIt script (that actually fully works for a couple
Active Members
now). All I wish is to make text grabbing more reliable.
4
 
598 posts

LarsJ Posted October 24, 2019

The Cool Code Company
You can easily manipulate and edit an image and extract text from an image. Rows and columns in an image are just a certain number of
horizontal and vertical directions.
 
With classic automation code you can automate this set of controls and with UI automation code you can automate this set of controls. Th
such controls as console or terminal controls.

MVPs

 1,118
1,822 posts File Explorer,  ROT objects,  UI Automation,  Windows Message Monitor
Compiled code: Accessing AutoIt variables,  DotNet.au3 UDF,  Using C# and VB code

Shell menus: The Context menu,  The Favorites menu.  Shell related: Control Panel,  System Image Lists

https://www.autoitscript.com/forum/topic/196833-uiaspy-ui-automation-spy-tool/page/4/ 4/8
4/24/2021 UIASpy - UI Automation Spy Tool - Page 4 - AutoIt Example Scripts - AutoIt Forums
Graphics related: Rubik's Cube,  OpenGL without external libraries,  Navigating in an image,  Non-rectangular selections
Control related: Hot-Track Enabled Toolbar Menu,  Simulating a modeless Choose Color Dialog,  Colors and fonts in TreeViews
ListView controls: Colors and fonts,  Multi-line header,  Multi-line items,  Checkboxes and icons,  Incremental search
ListView controls: Virtual ListViews,  Editing cells,  Data display functions

supersonic Posted October 24, 2019 (edited)

Universalist
@LarsJ  Thank you - Good to know. I'll try to keep that in mind...

S
Edited October 24, 2019 by supersonic

Active Members
4
598 posts

junkew Posted October 24, 2019

Universalist
https://www.mochasoft.dk/tn5250menus.htm

Shows you can copy paste as text or csv

1. WinActivate the mocha window main window

2. Click xy counted from xy of the window to be in the terminal area

MVPs

3. Send home to be at terminal position 0 0

 407
2,648 posts
4. Send shift arrow down 24 times and shift arrow right 80 or thru menu send select all or just ctrl a

5. Send copy either thru menu or with shortcut key

6. Clipboard functions AutoIt should be able to get it from the keyboard.

Doing 1 and 2 should be 100% reliable to make sure you are in terminal area.

Just post simplespy output based on that we can help you probably a little further at least there is a mainwindow child of desktop.

FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets

junkew Posted October 25, 2019

Universalist
Seems to be a full GDI black area with drawtext happening.

Control is not detectable so just calculate position based on position info of main info and show that highlighted area and activate it with a

Example (long time not played with an as400 or other IBM mainframe  ;-))

1. Download and install mocha tn5250

MVPs
2. Connect to www.pub400.com (either with or without userid, logonscreen will come allways)
 407 3. Run below script (after you have done le, edit session, connect,) I did not automate the menus as thats straight forward with alt an
2,648 posts
Some calculation has to be done based on the font size you have installed
Mocha is a limited terminal emulator, no way to easy nd out the blinking cursor position and no x, y in toolbar

Functions are generic

setPosTN5250 with x, y will calculate position and with mouseclick cursor will be positioned
getTN5250Text will just return the text in the black area
getTN5250hwnd will determine hwnd of rst windows10.* with text toolstrip1, .... 
selectBlockTN5250 will select a rectangle of text (will not copy it thats just another send ctrl c)
_UIA_DrawRect draws a rectangle 😂

Script

 
 ex

https://www.autoitscript.com/forum/topic/196833-uiaspy-ui-automation-spy-tool/page/4/ 5/8
4/24/2021 UIASpy - UI Automation Spy Tool - Page 4 - AutoIt Example Scripts - AutoIt Forums

#include <WinAPI.au3>
#include <WindowsConstants.au3>
#include <Array.au3>
;~ test with mocha tn5250
;~ www.pub400.com

global $g_baseLeft
global $g_baseRight
global $g_baseTop
global $g_baseBottom
global $g_TN5250Window

Example()

func example()
$g_TN5250Window=getTN5250hwnd()
WinActivate($g_TN5250Window,"")
Local $aPos = WinGetPos($g_TN5250Window)
;~ $aPos[0] = X position
;~ $aPos[1] = Y position
;~ $aPos[2] = Width
;~ $aPos[3] = Height
;~ Left,right,top,bottom
_UIA_DrawRect($aPos[0],$aPos[0]+$aPos[2],$aPos[1],$aPos[1]+$aPos[3])

;~ So some calculations for the black area

$g_baseLeft=$aPos[0]+8
$g_baseRight=$aPos[0]+$aPos[2]-24
$g_baseTop=$aPos[1]+78
$g_baseBottom=$aPos[1]+$aPos[3]-40
_UIA_DrawRect($g_baseLeft,$g_baseRight,$g_baseTop,$g_baseBottom)

;~ Lets positon mouse over the first row

for $iCol=1 to 80
setPosTN5250($iCol,0)
Next

L t it th fi t l
 

FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets

LarsJ Posted November 21, 2019

The Cool Code Company
UIASpy updates to Windows 10 version 1809
Documentation for Windows 10 version 1809 updates can be found in Windows 8, Windows 8.1 and Windows 10 updates. This post is pre
completely rewritten.

In addition, there are a few minor updates that are documented below.
MVPs
A small but signi cant update is that in Sample code to nd the application top window, $TreeScope_Descendants is replaced by
 1,118 $TreeScope_Children. The application top window is always a direct child of the Desktop. See the treeview in UIASpy. Therefore, this updat
1,822 posts made. This is a major performance update when the application window is not at the top of the Z-order of windows.

UIA_MouseClick() in UIA_Functions.au3 is extended with right-click functionality. Sample code can be generated through Sample code |
snippets ...

In UIASpy.ini, through the DisplayScale key, you can specify the display scale you want when UIASpy starts up.

Still not implemented update: Integrating $PropertyConditionFlags_MatchSubstring (Windows 10 1809) into Sample code.

New zip- le at bottom of rst post.

Euzguer and argumentum

File Explorer,  ROT objects,  UI Automation,  Windows Message Monitor

Compiled code: Accessing AutoIt variables,  DotNet.au3 UDF,  Using C# and VB code
Shell menus: The Context menu,  The Favorites menu.  Shell related: Control Panel,  System Image Lists
Graphics related: Rubik's Cube,  OpenGL without external libraries,  Navigating in an image,  Non-rectangular selections
Control related: Hot-Track Enabled Toolbar Menu,  Simulating a modeless Choose Color Dialog,  Colors and fonts in TreeViews
ListView controls: Colors and fonts,  Multi-line header,  Multi-line items,  Checkboxes and icons,  Incremental search
ListView controls: Virtual ListViews,  Editing cells,  Data display functions

argumentum Posted November 21, 2019

✨Universalist ✨
#include <WinAPISysWin.au3>  shows missing for AutoIt v3.3.14.5

Thanks for the update

https://www.autoitscript.com/forum/topic/196833-uiaspy-ui-automation-spy-tool/page/4/ 6/8
4/24/2021 UIASpy - UI Automation Spy Tool - Page 4 - AutoIt Example Scripts - AutoIt Forums

Lars

Active Members
 643 Follow the link to see my signature's stu .
2,184 posts

LarsJ Posted November 21, 2019

The Cool Code Company
Thank you Mr. argumentum. I'm on AutoIt 3.3.14.2 and forgot to test on 3.3.14.5. Zip- le in both examples is updated.

argumentum

MVPs
File Explorer,  ROT objects,  UI Automation,  Windows Message Monitor
Compiled code: Accessing AutoIt variables,  DotNet.au3 UDF,  Using C# and VB code
 1,118
Shell menus: The Context menu,  The Favorites menu.  Shell related: Control Panel,  System Image Lists
1,822 posts
Graphics related: Rubik's Cube,  OpenGL without external libraries,  Navigating in an image,  Non-rectangular selections
Control related: Hot-Track Enabled Toolbar Menu,  Simulating a modeless Choose Color Dialog,  Colors and fonts in TreeViews
ListView controls: Colors and fonts,  Multi-line header,  Multi-line items,  Checkboxes and icons,  Incremental search
ListView controls: Virtual ListViews,  Editing cells,  Data display functions

 PREV 1 2 3 4 5 NEXT  Page 4 of 5  

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account Sign in

Sign up for a new account in our community. It's easy! Already have an account? Sign in here.

Register a new account Sign In Now

 GO TO TOPIC LISTING

   

Recently Browsing   0 members

No registered users viewing this page.

 Home  AutoIt v3  AutoIt Example Scripts  UIASpy - UI Automation Spy Tool 

https://www.autoitscript.com/forum/topic/196833-uiaspy-ui-automation-spy-tool/page/4/ 7/8
4/24/2021 UIASpy - UI Automation Spy Tool - Page 4 - AutoIt Example Scripts - AutoIt Forums
Theme  Privacy Policy Contact Us
Powered by Invision Community

https://www.autoitscript.com/forum/topic/196833-uiaspy-ui-automation-spy-tool/page/4/ 8/8