Announcing the WhiteboxTools library


Things have been moving forward with Whitebox GAT at a fast pace recently. I have had numerous request over the years from people who want to use Whitebox GAT tools from outside of the user interface (UI). While all of Whitebox’s tools can be called from scripts, historically, there has been no way to call a Whitebox tool from outside the UI. Quite some time ago, I committed myself to finding a solution for this problem but efforts have been delayed by a coincident urgent development need. Increasingly the decision to develop Whitebox GAT in Java has been problematic as systems provide ever more marginal support for the Java platform. The solution to these two issues (i.e. external automation of Whitebox tools and the need to decrease reliance on Java) has a convenient synergistic overlap. Today I am publicly announcing the WhiteboxTools library. WhiteboxTools is a sub-project of the Whitebox GAT open-source GIS project. A description below is taken from the WhiteboxTools website:

WhiteboxTools is an advanced geospatial data analysis engine. The library has been developed using the Rust programming language, a very performant and safe systems language often viewed as a modern replacement for C/C++. Although WhiteboxTools is intended to serve as a source of plugin tools for the Whitebox GAT open-source GIS project, the tools contained in the library are stand-alone and can run outside of the larger Whitebox GAT project. See Usage for further details. There have been a large number of requests to call Whitebox GAT tools and functionality from outside of the Whitebox user-interface (e.g. from Python automation scripts). WhiteboxTools is intended to meet these usage requirements. Eventually most of the approximately 400 tools contained within Whitebox GAT will be ported to WhiteboxTools. In addition to separating the processing capabilities and the user-interface (and thereby reducing the reliance on Java), this migration should significantly improve processing efficiency. This is because Rust is generally faster than the equivalent Java code and because many of the WhiteboxTools functions are designed to process data in parallel wherever possible. In contrast, the older Java codebase included largely single-threaded applications.”

At the time of writing, approximately 170 of Whitebox GAT’s plugins have been added to the WhiteboxTools library and more are added every day. WhiteboxTools is already a powerhouse of geospatial analysis in a small 5MB package! The development to date has focused on porting Whitebox’s extensive raster and LiDAR analysis toolset and vector support will eventually be added as well. Distributed under the highly permissive MIT license, WhiteboxTools can be embedded into other software projects to provide geospatial analytical capabilities. The library is currently considered experimental; as the project stabilizes, its functionality will be incorporated into the larger Whitebox GAT project itself. Feedback and contributions are always welcome.


9 thoughts on “Announcing the WhiteboxTools library

  1. Neelum says:

    This is really another intriguing aspect of WB. I have found this software as one of the most reliable tools to perform advance geospatial analysis. I appreciate the continuity of efforts being employed for further enhancement in the software. I can anticipate that all geospatial community will take advantage by availability of its tool library. The efforts rendered by the developers is appreciable.

  2. HeathenCleric says:

    Python, Groovy, Javascript, Go and now Rust…

    You are giving potential contributors a hard time, especially in the Python-centric GIS world 😉

    And I’m not sure if performance should have been the main priority in selecting the language, if the tools keep being limited to the amount of available system memory.

    But for users, the ability to use those awesome tools in any Python script is invaluable!

    Really looking forward to 1.0!

    And thank you for your amazing work!

    • Actually, porting the existing tools to Rust doesn’t preclude people from contributing or writing custom tools in the scripting language of their choice (i.e. Python, Groovy and Javascript will still be supported for in-program scripting). As for the issue of system memory, natively compiled programs require far less memory than Java does; so this is just one more win for Rust vs the old Java tools. The reduction in memory usage is really quite significant. You should find that less system memory is required by these new tools compared with the old tools and therefore you should be able to process even larger files.

      The Python-centric GIS world is one of the motivating factors in making this move. I know that there are a lot of users out there that want to be able to call Whitebox’s tools from CPython rather than the Jython (Python on the Java runtime) that is embedded in Whitebox GAT. This will allow these users to do that. I’m hoping that this will actually encourage contributions rather than discourage 😉

  3. André Kruger says:

    Wow! This is very exciting. I use Python from Jupyter Notebook as my main GIS working environment. So far most of my raster commands was done through saga_cmd. This will definitely become part of my workflow.

    Thank You for your hard work.

  4. Emily Mills says:

    WhiteboxTools is an amazing resource. This will be a game-changer in terms of automating processes. I would love to use this to automate breaching DEMs over large geographic areas. Is there any plan to incorporate the regular “Breach Depressions” tool from GAT into WhiteboxTools? The current outputs using python scripting look like the “Breaching Depressions-
    Fast” output.

  5. William Lidberg says:

    How should we refer to whitebox tools in publications? Is “The Whitebox Geospatial Analysis Tools Project and Open-Access GIS” still the recommended article?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s