Debugging and Interactive Development of Time Cockpit Python Scripts

It is easy to create small scripts to automate tasks or extend time cockpit's functionality. When the requirements and scripts grow more complex step-debugging and a REPL are desirable features we do not (yet) provide within time cockpit. This post shows how these features can be set up using Visual Studio or other development environments. The full sample is available for download and possible updates can be found at our github repository.

Prerequisites

The following tools need to be installed in order to run the samples in this post.

Time Cockpit

In order to access data from time cockpit (version 1.9 at the time of this writing) it has to be installed and configured. Note that you could also create a similar environment for other applications providing a .NET SDK which can be used from IronPython.

IronPython

The IronPython installer (version 2.7.3 at the time of this writing) will setup the tools to run Python scripts on the .NET-runtime. It is a good idea to watch out for matching IronPython versions installed on the system and shipped with time cockpit. This is no strict constraint but there might be issues in more complex scenarios.

Visual Studio 2010 or 2012

Visual Studio has been chosen as the IDE for this example. There are many Python-IDEs with support for IronPython which should work as an alternative. Examples are #develop, PyDev or Wing IDE. The samples have been created using VS 2012 but VS 2010 should work as well.

Python Tools for Visual Studio

The Python Tools for Visual Studio (version 1.5 at the time of this writing) add support for Python to different parts of the IDE.

Sample python script

Creating a fully working python file which can be debugged and is capable of using the time cockpit SDK involves some boilerplate code. We are working on reducing the required steps for future versions of our SDK. Many of the steps might also be relevant for other .NET application SDKs.

Basic Configuration

We first set up some basic configuration variables. They define if the client or server database should be used, where the time cockpit binaries are located and where the log-file should be written to.

from System import Environment

# Configuration
timeCockpitLocation = Environment.ExpandEnvironmentVariables(r"%ProgramW6432%\software architects\time cockpit\time cockpit 2010")
logFileName = r"python.log"

App.config Handling

As explained in another post a library might depend on a companion app.config file. The following shows the required utility class and the setup required for time cockpit.

# Configuration file handling
import clr
clr.AddReference("System.Configuration")
from System.Configuration.Internal import IInternalConfigSystem
class ConfigurationProxy(IInternalConfigSystem):
    def __init__(self, fileName):
        from System import String
        from System.Collections.Generic import Dictionary
        from System.Configuration import IConfigurationSectionHandler, ConfigurationErrorsException
        self.__customSections = Dictionary[String, IConfigurationSectionHandler]()
        loaded = self.Load(fileName)
        if not loaded:
            raise ConfigurationErrorsException(String.Format("File: {0} could not be found or was not a valid cofiguration file.", fileName))

    def Load(self, fileName):
        from System.Configuration import ExeConfigurationFileMap, ConfigurationManager, ConfigurationUserLevel
        exeMap = ExeConfigurationFileMap()
        exeMap.ExeConfigFilename = fileName
        self.__config = ConfigurationManager.OpenMappedExeConfiguration(exeMap, ConfigurationUserLevel.None)
        return self.__config.HasFile
    
    def GetSection(self, configKey):
        if configKey == "appSettings":
            return self.__BuildAppSettings()
        return self.__config.GetSection(configKey)
    
    def __BuildAppSettings(self):
        from System.Collections.Specialized import NameValueCollection
        coll = NameValueCollection()
        for key in self.__config.AppSettings.Settings.AllKeys:
            coll.Add(key, self.__config.AppSettings.Settings[key].Value)
        return coll

    def RefreshConfig(self, sectionName):
        self.Load(self.__config.FilePath)
        
    def SupportsUserConfig(self):
        return False
    
    def InjectToConfigurationManager(self):
        from System.Reflection import BindingFlags
        from System.Configuration import ConfigurationManager
        configSystem = clr.GetClrType(ConfigurationManager).GetField("s_configSystem", BindingFlags.Static | BindingFlags.NonPublic)
        configSystem.SetValue(None, self)

References and Imports

The next step is loading the necessary DLLs, importing types and setting up LINQ.

# References
clr.AddReferenceToFileAndPath(Path.Combine(timeCockpitLocation, "TimeCockpit.Data.dll"))
clr.AddReference("TimeCockpit.Common")
clr.AddReference("TimeCockpit.UI.Common")
clr.AddReference("System.Core")
import System
from TimeCockpit.Common import Logger, LogLevel
from TimeCockpit.UI.Common import TimeCockpitApplication, DataContextConnections, ConnectionType
clr.ImportExtensions(System.Linq)

Setting up the DataContext

To access the client or server database of your time cockpit installation the correct DataContext has to be set up. We also get the required settings from time cockpit's configuration and enable logging.

# time cockpit DataContext
TimeCockpitApplication.Current.InitializeSettings()
Logger.Initialize(logFileName, TimeCockpitApplication.Current.ApplicationSettings.MinimumLogLevel)
TimeCockpitApplication.Current.InitializeDataContext(False, True)
connection = DataContextConnections.Current.Single(lambda dcc: dcc.ConnectionType == ConnectionType.ApplicationServer)
connection.InitializeOrThrow(False)
Context = connection.DataContext

Main Script Content

Now we are good to go. An environment very similar to the built-in script editor has been set up which can now be run in Visual Studio, other IDEs or stand-alone IronPython. The only difference is that the set of implicitly available imports has been simplified (the download contains the full and simplified versions).

In this sample we query all projects, iterate over them and print them in different ways depending on some condition.

projects = Context.Select("From P In Project Select P")
for p in projects:
    if p.ProjectName.Contains('c'):
        print "!!", p.ProjectName, p.StartDate
    else:
        print p.ProjectName, p.StartDate

Debugging

Our self-contained time cockpit python script can be debugged in different ways. If you just want to use our SDK a python-based approach like the Standard Python launcher in PyTools works best. This shows the Python-perspective of all objects, allows to use the watch window and similar inspection mechanisms. If you would like to debug and step into .NET code called from within the python script you want to use the IronPython (.NET) launcher. There is currently no mixed-mode debugger in PyTools which would allow to transparently debug both kinds of code.

Breakpoints

Breakpoints can be used to suspend execution. PyTools should also support conditional breakpoints, but I was unable to use them in the current version.

Inspect and Watch

If you quickly want to have a look at the data in your script, you can hover over a variable or a selected member.

A highlighted expression may even contain function calls or other non-trivial constructs. In some cases this might cause side effects and should be used with care.

The watch window can be used for a more permanent look on certain variables or expressions.

Interactive Querying, Debugging and Developing

While developing a script it is often desirable to try queries and functions in a prototypical and interactive way. This is where interactive debug windows and REPLs might be useful. A script might take some time to reach a certain state which is required to develop the next steps. It is quite cumbersome to launch and run the whole script after every small addition.

Interactive Debugging

Let's assume we defined some functions and selected some data. In order to quickly try a new function the existing script can be run and suspended via a breakpoint at the end.

def isRelevant(project):
    return project.Code.Contains('test')

def printRelevantProjects(projects, isRelevant):
    for p in projects.Where(isRelevant):
        print p.ProjectName, p.StartDate

projects = Context.Select("From P In Project Select P")
# TODO complete me

We now like to test the functions and try possible variations using the Python Debug Interactive window.

This allows us to write ad-hoc Python code with some completion and type information.

After running the function we notice that the result is empty which was not expected.

The bug is located in the check within the isRelevant filter function. We can fix and redefine the function implementation and run it again. The expected result is now printed.

Depending on the provided type information, additional completion options on .NET types might be available to further ease interactive development.

Using the PyTools REPL

If we do not want to start off from a debugging session, the IronPython Interactive window (see VIEW/Other Windows/IronPython 2.7 Interactive) can be used. It provides an empty scope and has some helper commands to load external scripts, change settings or attach the debugger.

In order to set up the data context we load the preamble python script.

The interactive shell will execute the script as if it was typed in by the user.

Now that the data context is available we can query some data and take a closer look at it. In the following sample all existing users are selected and kept in a variable. Then a LINQ extension method for finding a single object matching a specific predicate is executed on the collection of users. The red exception text, hints that there are several users having Simon as their first name. If we refine the condition to exclude hidden users the expected result is found and a single object returned to be printed.

Conclusion

This article shows some examples on how to simplify development of Python scripts with a special focus on IronPython and time cockpit. There are still many situations where the tools and consumed .NET libraries could improve the amount of available information and support. Even in its current state interactive development of queries, functions or complete scripts can be useful especially if the run-time of the scripts grows longer.