As I found some lessons learning while coding and debugging my Python workbench, I would like to share my experience and collect some further advise, with the ultimate goal to improve the documentation (and make good use of the advice myself). So you may well assume that this topic is intermediate between the "Developers corner" and "Wiki", and you will excuse me for selecting "Developers" as I believe that the post has more relevance here for the initial discussion.
Also, Python workbenches debugging is strictly related also to Python workbenches reloading. So I'm summarizing here my understanding about that as well.
Let's start with debugging. Python debugging is actually described in two WIki pages:
https://www.freecadweb.org/wiki/Python_ ... nvironment
-> this I find quite obsolete, as it basically describes how you can debug Python code using 'print' statements or slightly more elaborated versions of the same concept. Interesting, but not exactly user-friendly
However, much more information can be found searching in the Forum, but in the end (as these are Forum questions) it is not well structured and sometimes also misleading, as spread over many years and different FreeCAD versions. Also, most of it deals on how to debug Macros, not specifically Modules/Workbenches.
So here's my summary:
Method 1: Using
pdb
This is my preferred, as it works out of the box (requires no installation of anything beyond the standard FreeCAD release), it is the same for Windows and Linux, and can debug pretty much everything Python.
Basic usage for pdb in FreeCAD is discussed in
https://forum.freecadweb.org/viewtopic.php?f=10&t=28256
However, the suggestion is to insert the statement
in your
code
, instead of
print
statements. This is not quite right, as in general I do not want to change the code for debugging; and more so when dealing with workbenches, where I fall into the secondary problem on how to reload a workbench!
Instead, as we have the Python console available, from the console you just:
Now when FreeCAD hits the breakpoint, pdb will kick-in in the Python console, and you can inspect/change variables, arguments, execute Python statements, etc.
Another advantage of pdb is that it allows you post-mortem analysis. If you hit an exception, you can run from the Python console
and you have access at the last backtrace, so you can dig into what happened.
Cons of this approach:
It is textual only (well, for me this is not necessarily a cons)
FreeCAD mess up a bit the prompt (pdb) when debugging on a breakpoint, the prompt appears sometimes in the report window, so it might be a bit confusing
Method 2: using
winpdb
As the name implies, this is a GUI version of pdb (not a Windows-only version, mind you! win=GUI here). However, to make it work, you need some configuration, which is different for Windows and Linux, as of course you need to install winpdb, and the debug is done attaching winpdb to a remote target (is not embedded within FreeCAD; FreeCAD is the remote target to debug).
This is the method described by Werner in many posts (I believe this is his preferred?), see:
https://forum.freecadweb.org/viewtopic. ... 80&p=26446
https://forum.freecadweb.org/viewtopic.php?f=10&t=3884
and ultimately appearing in the Wiki in the post I already cited:
https://www.freecadweb.org/wiki/Debugging
Again all the discussion is mainly for debugging Macros, and the example in the Wiki page tells you to run a script. Actually this is not needed, again you can just insert a breakpoint in your code and work in FreeCAD until that part of the code is reached.
Linux version:
in winpdb, pause the execution (FreeCAD becomes unresponsive)
in winpdb, open your source file, and set the breakpoint
in winpdb, resume execution
Note that installing winpdb automatically installs rpdb2 in your Python environment.
Windows version:
The problem under Windows is that Python for FreeCAD is installed bundled with FreeCAD, and separately from any other Python installation you may have.
So when you need to install Python extensions, you need to do that in a way that FreeCAD sees it.
However, you don't need winpdb to be installed into FreeCAD; this application opens a socket for remote debugging, so you can launch it in any other way. In my case, as I have Python installed under Win, I just installed it from the DOS shell using:
However, from within FreeCAD you need rpdb2. I looked around a bit in the Forum and found the discussion
https://forum.freecadweb.org/viewtopic.php?t=16851
. So I gave it a try and it actually worked, at least in my case - the thread warns that this method may fail, I'm not sure here about the conditions when this could happen.
One important remark: if you have FreeCAD installed in "Program Files" or equivalent directory that is under Admin supervisory right, this method of installation will fail. You MUST start FreeCAD with "Run as Administrator".
Now you are all set, and you can follow similar steps as in Linux case:
start winpdb. Go to menu File->Password" and set the password to e.g. "test"
in winpdb, pause the execution (FreeCAD becomes unresponsive)
in winpdb, open your source file, and set the breakpoint
in winpdb, resume execution
The obvious advantage of winpdb is the availability of a GUI, so it is more user-friendly than pdb alone. However, I find it more tricky to start (as you need to work on two different windows, coordinating the actions) and the author of winpdb says
Cons of this approach:
Needs a more complex set-up (at least under Windows)
More complex to start than pdb (as you need to work on two different windows, coordinating the actions)
Method 3: using
Visual Studio
This is described in detail in the thread
https://forum.freecadweb.org/viewtopic.php?f=22&t=28901
However, I admit I was not able to make it works, possibly because (for some reasons not relevant here) I have only VS2015 installed.
But besides that:
Cons of this approach:
Only Windows
Tied to Visual Studio, that is fully proprietary (I know that FreeCAD under Win compiles under VS, but whenever I can, I prefer open tools)
Method 4: using
FreeCAD embedded Python debugger/editor
This would be the preferred method, but actually the debugger is not documented (or at least I could not find the relevant Wiki page), and I could find no way to make it work for debugging a workbench.
So your option here is creating a stub i.e. a Macro that calls the functions of your workbench that you need to debug, so that you can load the macro in the editor, set breakpoints and run it.
Cons of this approach:
Not really suited to debug workbenches, only macros
Reloading workbenches
Ok, so now you debugged your code, found the issue and fixed it. How can you reload your workbench without closing/re-opening FreeCAD?
There are two main posts that deal with the topic:
https://forum.freecadweb.org/viewtopic.php?t=320
https://forum.freecadweb.org/viewtopic. ... 5&p=296289
First of all a note: as far as my understanding and my experience goes, you cannot really reload a full workbench, as there are some functions that loaded by C/C++ code. In particular:
the Workbench description class is transfered to FreeCAD while start
FreeCADGui.addCommand('MyModule', MyModule())
To reload everything else, you can use in Python 2:
import someModule
from importlib import reload
reload(someModule)
Now a word of caution. What can make this method fail is forgetting about the namespaces you are in. In particular, if your workbench definition file, say someModule.py, imports other files with
from someTool import *
then you are actually importing the definitions in the namespace someModule.
So while in general you can import * and even reload it, with
import someTool
reload( someTool )
from someTool import *
this is not possible from the Python console as you are not in the correct namespace (someModule)
(note: hint available at last row of the text appearing when typing import this, but probably you all already know that).
What you can do (this was suggested by Microelly I believe) is to tweak your someModule to contain, instead of the plain from someTool import *, the full code above.
Then you just need to import someModule and then reload someModule and the trick is done.
I'm not sure about the performance impact of leaving the full reload code in someModule in a production release of the workbench, but at least you can leave it there until you are done with your testing.
I realize I made an extra-long post, sorry about that and thanks for reading so far. I would welcome everybody's comments, as soon as this is cleared up I will translate the post into the Wiki pages, ideally marking https://www.freecadweb.org/wiki/Python_ ... nvironment as old, while improving https://www.freecadweb.org/wiki/Debugging .
Of course debugging Macros, and not workbenches, is a subset of workbench debugging, and considerably easier.
Ciao!
Enrico
Awesome summary!
This needs to be part of FreeCAD #documentation because debugging can save a ton of time if you know how use it.
@kkremitzki recently turned me on to pdb and it's seriously awesome.
There are also cool refinements that people should know about, there is a dropin replacement for pdb called pdbplusplus (pdbpp) that has an awesome sticky mode. I learned about it from this fantastic pdb intro talk back from pycon 2015:
Watching the whole thing is well worth it.
Actually, I think it would be an amazing gift if we had a GSOC project that was dedicated to integrating a debugger (continuing @mumme's awesome work in @ediloren's mentioned thread)
I will post a really newbie question here as I'm new booth to FreeCAD macro's development and its internal api and to python programming. Is it possible to open a macro file or a code from a workspace from inside an IDE like intellij and debug it from there? (I use to do this kind of thing with nodejs, php on a browser, javascript on a browser, java as a remote connection and so on)
@Kunda1 : thanks for the pointer to pdbpp, I did not know that. However I am not sure to which extent we can use it in FreeCAD, as it actually tries to repaint the screen (this is the whole purpose of the sticky mode), color the output etc. so I wonder how's that in the FreeCAD console. I am not testing it for the time being, as it replaces pdb.py, so I don't want to mess up the installation when I need to 'pip uninstall' (probably nothing bad will happen, but I am not ready for a test now). If anybody is giving that a shot, I'd welcome knowing the results (both for Windows as well as for Linux).
@iogui : I do not use intellij, but what you ask for is possible, provided you have the right workspace. Actually this is exactly method 3 in my list, where the workspace IDE is Visual Studio (which is quite powerful, to be frank - only I don't like to stick to proprietary closed-source software that may change the license later on; plus (or minus..) this is only for Windows). Other IDEs may be able to attach to a remote Python process for debugging, just like VS does, or using rpdb as winpdb does (which works also through an LAN connection), but I am not able at present to suggest you other options besides VS.
Enrico
ediloren wrote: ↑Wed Apr 03, 2019 9:13 pm
@iogui : I do not use intellij, but what you ask for is possible, provided you have the right workspace. Actually this is exactly method 3 in my list, where the workspace IDE is Visual Studio (which is quite powerful, to be frank - only I don't like to stick to proprietary closed-source software that may change the license later on; plus (or minus..) this is only for Windows). Other IDEs may be able to attach to a remote Python process for debugging, just like VS does, or using rpdb as winpdb does (which works also through an LAN connection), but I am not able at present to suggest you other options besides VS.
Well Intellij is quite powerful too and has a community version that is open-source indeed and it works nicely on windows, mac or linux. As I am a Linux only user I do prefer it over Visual Studio and I also own an Intellij commercial license. I have seen that Intellij has a remote debugging on python option but I have never tried it. I've made a small search in my environment to find their "pycharm-debug-egg" python file but I haven't found nothing. It is really nice to have an IDE editing and debugging tool as you develop something so it would be useful. Unfortunately, I think it will not work with intellij... sad...
I successfully managed to use PyCharm for debugging a workbench script. Remote debugging is indeed not available in the Community Edition and that might be the reason why you do not find the egg.
However you don't need remote debugging as long as PyCharm and the FreeCAD application are running on the same machine. You can just attach the remote debugger to a running FreeCAD instance and off you go. In PyCharm this can be found under Run / Attach To Process or with Ctrl-Alt-F5. You are presented with a list of processes to select from. You can configure the processes to appear in this list under File / Settings / Build, Execution, Deployment / Python Debugger. There you can specify a string. All the processes containing this string in their name will appear in the process list to select from later on. The initial search string is "python" such that FreeCAD will not appear in the list. Changing it to "FreeCAD" should work. When in doubt, inspect the list of running processes with the tools of your OS and also note the process ID.
I don't know whether and how you are able to access this functionality when using the IntelliJ plugin instead of the standalone PyCharm application, but it might be worth having a look around the menu.
hlg wrote: ↑Fri Jul 12, 2019 10:30 am
Remote debugging is indeed not available in the Community Edition and that might be the reason why you do not find the egg.
No it isn't. I'm not using the community edition. I'm using the ultimate edition of Intellij that has indeed the python plugin.
hlg wrote: ↑Fri Jul 12, 2019 10:30 am
You can just attach the remote debugger to a running FreeCAD instance and off you go.
Ok. I don't know if it will work, but I will give it a try later when I have the time to do it.
hlg wrote: ↑Fri Jul 12, 2019 10:30 am
I successfully managed to use PyCharm for debugging a workbench script. Remote debugging is indeed not available in the Community Edition and that might be the reason why you do not find the egg.
However you don't need remote debugging as long as PyCharm and the FreeCAD application are running on the same machine. You can just attach the remote debugger to a running FreeCAD instance and off you go. In PyCharm this can be found under Run / Attach To Process or with Ctrl-Alt-F5. You are presented with a list of processes to select from. You can configure the processes to appear in this list under File / Settings / Build, Execution, Deployment / Python Debugger. There you can specify a string. All the processes containing this string in their name will appear in the process list to select from later on. The initial search string is "python" such that FreeCAD will not appear in the list. Changing it to "FreeCAD" should work. When in doubt, inspect the list of running processes with the tools of your OS and also note the process ID.
Hi hlg
I was able to follow your instructions with pycharm and to create a connection with FreeCad.
But, now? I`m on there very beginning of scripting and coding and have no idea how to get a workbench debugged?
What would be the next steps to get the code into PyCharm?
Thanks for help.
Chris
I open FreeCAD. I enter the following into the console to load my version of Arch workbench (especially ArchWindow module)
import importlib # to use reload() later
sys.path.insert(0, "/home/steelman/src/FreeCAD/src/Mod/Arch")
for m in [k for k in sys.modules.keys() if 'Arch' in k]: sys.modules.pop(m, None)
import Arch
When I try to use pdb it either does not set a breakpoint
>>> import pdb
>>> pdb.set_trace()
--Return--
> <input>(1)<module>()->None
(Pdb) b ArchWindow:683
(Pdb) c
or it stops accepting any input except Enter key (which sets consequent copies of the breakpoint )
>>> import ArchWindow
>>> pdb.set_trace()
--Return--
> <input>(1)<module>()->None
(Pdb) b ArchWindow:683
Breakpoint 1 at /home/steelman/src/FreeCAD/src/Mod/Arch/ArchWindow.py:683
(Pdb) c
Breakpoint 2 at /home/steelman/src/FreeCAD/src/Mod/Arch/ArchWindow.py:683
(Pdb)
PS. To reload ArchWindow I am working on, I use the following commands:
sys.modules.pop('ArchWindow') # to unload, otherwise reloading Arch is no-op
importlib.reload(Arch)
EDIT:
Putting import pdb; pdb.set_trace() in the code works only slightly better. Although I can interact with pdb, it's really awkward because I write my commands in the console and get results in the report view. To make things worse pressing Enter sends a command to pdb but does not remove it from the command line.
