All Versions
91
Latest Version
Avg Release Cycle
54 days
Latest Release
932 days ago

Changelog History
Page 7

  • v0.3.23 Changes

    ๐Ÿš€ This release is the one that completes the Nuitka "sun rise phase".

    ๐Ÿš€ All of Nuitka is now released under Apache License 2.0 <http://www.apache.org/licenses/LICENSE-2.0>__ which is a very liberal license, ๐Ÿ†“ and compatible with basically all Free Software licenses there are. It's only asking to allow integration, of what you send back, and patent grants for the code.

    In the first phase of Nuitka development, I wanted to keep control over Nuitka, so it wouldn't repeat mistakes of other projects. This is no longer a concern for me, it's not going to happen anymore.

    I would like to thank Debian Legal team, for originally bringing to my ๐Ÿ‘ attention, that this license will be better suited, than any copyright assignment could be.

    ๐Ÿ› Bug fixes

    • The compiled functions could not be used with multiprocessing or copy.copy. Issue#19 <http://bugs.nuitka.net/issue19>__. Fixed in 0.3.22.1 already.

    • In-place operations for slices with not both bounds specified crashed the compiler. Issue#36 <http://bugs.nuitka.net/issue36>__. Fixed in 0.3.22.1 already.

    • Cyclic imports could trigger an endless loop, because module import expressions became the parent of the imported module object. Issue#37 <http://bugs.nuitka.net/issue37>__. Fixed in 0.3.22.2 already.

    • Modules named proc or func could not be compiled to modules or embedded due to a collision with identifiers of CPython2.7 includes. Issue#38 <http://bugs.nuitka.net/issue38>__. Fixed in 0.3.22.2 already.

    ๐Ÿ†• New Features

    • The fix for Issue#19 <http://bugs.nuitka.net/issue19>__ also makes pickling of compiled functions available. As it is the case for non-compiled functions in CPython, no code objects are stored, only names of module level variables.

    Organizational

    • Using the Apache License 2.0 for all of Nuitka now.

    • Speedcenter has been re-activated, but is not yet having a lot of benchmarks yet, subject to change.

    .. admonition:: Update

     We have given up on speedcenter meanwhile, and generate static pages with
     graphs instead.
    

    ๐Ÿ†• New Tests

    • ๐Ÿ”„ Changed the "CPython26" tests to no longer disable the parts that relied on copying of functions to work, as Issue#19 <http://bugs.nuitka.net/issue19>__ is now supported.

    • โœ… Extended in-place assignment tests to cover error cases of Issue#36 <http://bugs.nuitka.net/issue36>__.

    • โœ… Extended compile library test to also try and compile the path where numpy lives. This is apparently another path, where Debian installs some modules, and compiling this would have revealed Issue#36 <http://bugs.nuitka.net/issue36>__ sooner.

    Summary

    ๐Ÿš€ The release contains bug fixes, and the huge step of changing the license <http://www.apache.org/licenses/LICENSE-2.0>. It is made in preparation to PyCON EU <https://ep2012.europython.eu>.

  • v0.3.22 Changes

    ๐Ÿš€ This release is a continuation of the trend of previous releases, and added more re-formulations of Python that lower the burden on code generation and optimization.

    ๐Ÿ‘ It also improves Python3 support substantially. In fact this is the first ๐Ÿš€ release to not only run itself under Python3, but for Nuitka to compile itself with Nuitka under Python3, which previously only worked for Python2. For the common language subset, it's quite fine now.

    ๐Ÿ› Bug fixes

    • List contractions produced extra entries on the call stack, after they became functions, these are no more existent. That was made possible my making frame stack entries an optional element in the node tree, left out for list contractions.

    • ๐Ÿ‘ป Calling a compiled function in an exception handler cleared the exception on return, it no longer does that.

    • Reference counter handling with generator throw method is now correct.

    • A module "builtins" conflicted with the handling of the Python builtins module. Those now use different identifiers.

    ๐Ÿ†• New Features

    • ๐Ÿ†• New metaclass syntax for the class statement works, and the old __metaclass__ attribute is properly ignored.

    .. code-block:: python

     # Metaclass syntax in Python3, illegal in Python2
     class X(metaclass = Y):
         pass
    

    .. code-block:: python

     # Metaclass syntax in Python2, no effect in Python3
     class X:
         __metaclass__ = Y
    

    .. note::

     The way to make a use of a metaclass in a portable way, is to create a
     based class that has it and then inherit from it. Sad, isn' it. Surely, the
     support for ``__metaclass__`` could still live.
    
     .. code-block:: python
    
        # For Python2/3 compatible source, we create a base class that has the
        # metaclass used and doesn't require making a choice.
    
        CPythonNodeMetaClassBase = NodeCheckMetaClass(
            "CPythonNodeMetaClassBase",
            (object,),
            {}
        )
    
    • The --dump-xml option works with Nuitka running under Python3. This was not previously supported.

    • ๐Ÿ‘ป Python3 now also has compatible parameter errors and compatible exception error messages.

    • Python3 has changed scope rules for list contractions (assignments don't affect outside values) and this is now respected as well.

    • ๐Ÿ‘ Python3 has gained support for recursive programs and stand alone extension modules, these are now both possible as well.

    ๐Ÿ†• New Optimization

    • Avoid frame stack entries for functions that cannot raise exceptions, i.e. where they would not be used.

    This avoids overhead for the very simple functions. And example of this can be seen here:

    .. code-block:: python

     def simple():
        return 7
    
    • โšก๏ธ Optimize len built-in for non-constant, but known length values.

    An example can be seen here:

    .. code-block:: python

     # The range isn't constructed at compile time, but we still know its
     # length.
     len(range(10000000))
    
     # The string isn't constructed at compile time, but we still know its
     # length.
     len("*" * 1000)
    
     # The tuple isn't constructed, instead it's known length is used, and
     # side effects are maintained.
     len((a(), b()))
    

    This new optimization applies to all kinds of container creations and the range built-in initially.

    • โšก๏ธ Optimize conditions for non-constant, but known truth values.

    At this time, known truth values of non-constants means range built-in calls with know size and container creations.

    An example can be seen here:

    .. code-block:: python

     if (a,):
        print "In Branch"
    

    It's clear, that the tuple will be true, we just need to maintain the side effect, which we do.

    • โšก๏ธ Optimize or and and operators for known truth values.

    See above for what has known truth values currently. This will be most useful to predict conditions that need not be evaluated at all due to short circuit nature, and to avoid checking against constant values. Previously this could not be optimized, but now it can:

    .. code-block:: python

     # The access and call to "something()" cannot possibly happen
     0 and something()
    
     # Can be replaced with "something()", as "1" is true. If it had a side
     # effect, it would be maintained.
     1 and something()
    
     # The access and call to "something()" cannot possibly happen, the value
     # is already decided, it's "1".
     1 or something()
    
     # Can be replaced with "something()", as "0" is false. If it had a side
     # effect, it would be maintained.
     0 or something()
    
    • โšก๏ธ Optimize print arguments to become strings.

    The arguments to print statements are now converted to strings at compile time if possible.

    .. code-block:: python

     print 1
    

    becomes:

    .. code-block:: python

     print "1"
    
    • ๐Ÿ–จ Combine print arguments to single ones.

    When multiple strings are printed, these are now combined.

    .. code-block:: python

     print "1+1=", 1+1
    

    becomes:

    .. code-block:: python

     print "1+1= 2"
    

    Organizational

    • โœจ Enhanced Python3 support, enabling support for most basic tests.

    • ๐Ÿ‘• Check files with PyLint in deterministic (alphabetical) order.

    Cleanups

    • Frame stack entries are now part of the node tree instead of part of the template for every function, generator, class or module.

    • The try/except/else has been re-formulated to use an indicator variable visible in the node tree, that tells if a handler has been executed or not.

    • Side effects are now a dedicated node, used in several optimization to maintain the effect of an expression with known value.

    ๐Ÿ†• New Tests

    • โœ… Expanded and adapted basic tests to work for Python3 as well.

    • โž• Added reference count tests for generator functions throw, send, and close methods.

    • ๐Ÿ‘ป Cover calling a function with try/except in an exception handler twice. No test was previously doing that.

    Summary

    ๐Ÿš€ This release offers enhanced compatibility with Python3, as well as the solution to many structural problems. Calculating lengths of large non-constant values at compile time, is technically a break through, as is avoiding lengthy calculations. The frame guards as nodes is a huge improvement, making that โšก๏ธ costly operational possible to be optimized away.

    There still is more work ahead, before value propagation will be safe enough to ๐Ÿ‘€ enable, but we are seeing the glimpse of it already. Not for long, and looking at numbers will make sense.

  • v0.3.21 Changes

    ๐Ÿš€ This releases contains some really major enhancements, all heading towards enabling value propagation inside Nuitka. Assignments of all forms are now all simple and explicit, and as a result, now it will be easy to start tracking them.

    Contractions have become functions internally, with statements use temporary variables, complex unpacking statement were reduced to more simple ones, etc.

    ๐Ÿ›  Also there are the usual few small bug fixes, and a bunch of organizational ๐Ÿ‘Œ improvements, that make the release complete.

    ๐Ÿ› Bug fixes

    • The built-in next could causes a program crash when iterating past the end of an iterator. Issue#34 <http://bugs.nuitka.net/issue34>__. Fixed in 0.3.20.1 already.

    • The set constants could cause a compiler error, as that type was not considered in the "mutable" check yet. Fixed in 0.3.20.2 already.

    • ๐ŸŽ Performance regression. Optimize expression for exception types caught as well again, this was lost in last release.

    • Functions that contain exec, are supposed to have a writable locals. But when removing that exec statement as part of optimization, this property of the function could get lost.

    • The so called "overflow functions" are once again correctly handled. These once were left behind in some refactoring and had not been repaired until now. An overflow function is a nested function with an exec or a star import.

    • The syntax error for return outside of a function, was not given, instead the code returned at run time. Fixed to raise a SyntaxError at compile time.

    ๐Ÿ†• New Optimization

    • ๐Ÿ‘ป Avoid tuple objects to be created when catching multiple exception types, instead call exception match check function multiple times.

    • Removal of dead code following break, continue, return, and raise. Code that follows these statements, or conditional statements, where all branches end with it.

    .. note::

      These may not actually occur often in actual code, but future
      optimization may produce them more frequently, and their removal may in
      turn make other possible optimization.
    
    • Detect module variables as "read only" after all writes have been detected to not be executed as removed. Previously the "read only indicator" was determined only once and then stayed the same.

    • Expanded conditional statement optimization to detect cases, where condition is a compile time constant, not just a constant value.

    • โšก๏ธ Optimize away assignments from a variable to the same variable, they have no effect. The potential side effect of accessing the variable is left intact though, so exceptions will be raised still.

    .. note::

     An exception is where ``len = len`` actually does have an impact, because
     that variable becomes assignable. The "compile itself" test of Nuitka found
     that to happen with ``long`` from the ``nuitka.__past__`` module.
    
    • Created Python3 variant of quick unicode string access, there was no such thing in the CPython C/API, but we make the distinction in the source code, so it makes sense to have it.

    • โšก๏ธ Created an optimized implementation for the built-in iter with 2 parameters as well. This allows for slightly more efficient code to be created with regards to reference handling, rather than using the CPython C/API.

    • For all types of variable assigned in the generated code, there are now methods that accept already taken references or not, and the code generator picks the optimal variant. This avoids the drop of references, that e.g. the local variable will insist to take.

    • Don't use a "context" object for generator functions (and generator expressions) that don't need one. And even if it does to store e.g. the given parameter values, avoid to have a "common context" if there is no closure taken. This avoids useless malloc calls and speeds up repeated generator object creation.

    Organizational

    • ๐Ÿ”„ Changed the Scons build file database to reside in the build directory as opposed to the current directory, not polluting it anymore. Thanks for the patch go to Michael H Kent, very much appreciated.

    • The --experimental option is no longer available outside of checkouts of git, and even there not on stable branches (master, hotfix/...). It only pollutes --help output as stable releases have no experimental code options, not even development version will make a difference.

    • ๐Ÿšš The binary "bin/Nuitka.py" has been removed from the git repository. It was deprecated a while ago, not part of the distribution and served no good use, as it was a symbolic link only anyway.

    • The --python-version option is applied at Nuitka start time to re-launch Nuitka with the given Python version, to make sure that the Python run time used for computations and link time Python versions are the same. The allowed values are now checked (2.6, 2.7 and 3.2) and the user gets a nice error with wrong values.

    • โž• Added --keep-pythonpath alias for --execute-with-pythonpath option, probably easier to remember.

    • ๐Ÿ‘Œ Support --debug with clang, so it can also be used to check the generated code for all warnings, and perform assertions. Didn't report anything new.

    • 0๏ธโƒฃ The contents environment variable CXX determines the default C++ compiler when set, so that checking with CXX=g++-4.7 nuitka-python ... has become supported.

    • ๐Ÿ‘• The check-with-pylint script now has a real command line option to control the display of TODO items.

    Cleanups

    • ๐Ÿ”„ Changed complex assignments, i.e. assignments with multiple targets to such using a temporary variable and multiple simple assignments instead.

    .. code-block:: python

     a = b = c
    

    .. code-block:: python

     _tmp = c
     b = _tmp
     a = _tmp
    

    In CPython, when one assignment raises an exception, the whole thing is aborted, so the complexity of having multiple targets is no more needed, now that we have temporary variables in a block.

    All that was really needed, was to evaluate the complete source expression only once, but that made code generation contain ugly loops that are no more needed.

    • ๐Ÿ”„ Changed unpacking assignments to use temporary variables. Code like this:

    .. code-block:: python

     a, b = c
    

    Is handled more like this:

    .. code-block:: python

     _tmp_iter = iter(c)
     _tmp1 = next(_tmp_iter)
     _tmp2 = next(_tmp_iter)
     if not finished(_tmp_iter):
         raise ValueError("too many values to unpack")
     a = _tmp1
     b = _tmp2
    

    In reality, not really next is used, as it wouldn't raise the correct exception for unpacking, and the finished check is more condensed into it.

    Generally this cleanup allowed that the AssignTargetTuple and associated code generation was removed, and in the future value propagation may optimize these next and iter calls away where possible. At this time, this is not done yet.

    • ๐Ÿ‘ป Exception handlers assign caught exception value through assignment statement.

    Previously the code generated for assigning from the caught exception was not considered part of the handler. It now is the first statement of an exception handler or not present, this way it may be optimized as well.

    • ๐Ÿ‘ป Exception handlers now explicitly catch more than one type.

    Catching multiple types worked by merits of the created tuple object working with the Python C/API function called, but that was not explicit at all. Now every handler has a tuple of exceptions it catches, which may only be one, or if None, it's all.

    • Contractions are now functions as well.

    Contractions (list, dict, and set) are now re-formulated as function bodies that contain for loops and conditional statements. This allowed to remove a lot of special code that dealt with them and will make these easier to understand for optimization and value propagation.

    • ๐Ÿ— Global is handled during tree building.

    Previously the global statement was its own node, which got removed during the optimization phase in a dedicated early optimization that applied its effect, and then removed the node.

    It was determined, that there is no reason to not immediately apply the effect of the global variable and take closure variables and add them to the provider of that global statement, allowing to remove the node class.

    • Read only module variable detection integrated to constraint collection.

    The detection of read only module variables was so far done as a separate step, which is no more necessary as the constraint collection tracks the usages of module variables anyway, so this separate and slow step could be removed.

    ๐Ÿ†• New Tests

    • โž• Added test to cover order of calls for complex assignments that unpack, to see that they make a fresh iterator for each part of a complex assignment.

    • โž• Added test that unpacks in an exception catch. It worked, due to the generic handling of assignment targets by Nuitka, and I didn't even know it can be done, example:

    .. code-block:: python

     try:
         raise ValueError(1,2)
     except ValueError as (a,b):
         print "Unpacking caught exception and unpacked", a, b
    

    Will assign a=1 and b=2.

    • โž• Added test to cover return statements on module level and class level, they both must give syntax errors.

    • Cover exceptions from accessing unassigned global names.

    • โž• Added syntax test to show that star imports do not allow other names to be imported at the same time as well.

    • โœ… Python3 is now also running the compile itself test successfully.

    Summary

    The progress made towards value propagation and type inference is very significant, and makes those appears as if they are achievable.

  • v0.3.20 Changes

    ๐Ÿ›  This time there are a few bug fixes and some really major cleanups, lots of new optimization and preparations for more. And then there is a new compiler clang ๐ŸŽ and a new platform supported. macOS X appears to work mostly, thanks for the patches from Pete Hunt.

    ๐Ÿ› Bug fixes

    • The use of a local variable name as an expression was not covered and lead to a compiler crash. Totally amazing, but true, nothing in the test suite of CPython covered this. Issue#30 <http://bugs.nuitka.net/issue30>__. Fixed in release 0.3.19.1 already.

    • The use of a closure variable name as an expression was not covered as well. And in this case corrupted the reference count. Issue#31 <http://bugs.nuitka.net/issue31>__. Fixed in release 0.3.19.1 already.

    • The from x import * attempted to respect __all__ but failed to do so. Issue#32 <http://bugs.nuitka.net/issue32>__. Fixed in release 0.3.19.2 already.

    • The from x import * didn't give a SyntaxError when used on Python3. Fixed in release 0.3.19.2 already.

    • The syntax error messages for "global for function argument name" and "duplicate function argument name" are now identical as well.

    • Parameter values of generator function could cause compilation errors when used in the closure of list contractions. Fixed.

    ๐Ÿ†• New Features

    • โž• Added support for disabling the console for Windows binaries. Thanks for the patch go to Michael H Kent.

    • โœจ Enhanced Python3 support for syntax errors, these are now also compatible.

    • ๐Ÿ‘Œ Support for macOS X was added.

    • ๐Ÿ‘Œ Support for using the clang compiler was added, it can be enforced via --clang option. Currently this option is mainly intended to allow testing the "macOS X" support as good as possible under Linux.

    ๐Ÿ†• New Optimization

    • โœจ Enhanced all optimization that previously worked on "constants" to work on "compile time constants" instead. A "compile time constant" can currently also be any form of a built-in name or exception reference. It is intended to expand this in the future.

    • โž• Added support for built-ins bin, oct, and hex, which also can be computed at compile time, if their arguments are compile time constant.

    • โž• Added support for the iter built-in in both forms, one and two arguments. These cannot be computed at compile time, but now will execute faster.

    • โž• Added support for the next built-in, also in its both forms, one and two arguments. These also cannot be computed at compile time, but now will execute faster as well.

    • โž• Added support for the open built-in in all its form. We intend for future releases to be able to track file opens for including them into the executable if data files.

    • Optimize the __debug__ built-in constant as well. It cannot be assigned, yet code can determine a mode of operation from it, and apparently some code does. When compiling the mode is decided.

    • โšก๏ธ Optimize the Ellipsis built-in constant as well. It falls in the same category as True, False, None, i.e. names of built-in constants that a singletons.

    • โž• Added support for anonymous built-in references, i.e. built-ins which have names that are not normally accessible. An example is type(None) which is not accessible from anywhere. Other examples of such names are compiled_method_or_function.

    Having these as represented internally, and flagged as "compile time constants", allows the compiler to make more compile time optimization and to generate more efficient C++ code for it that won't e.g. call the type built-in with None as an argument.

    • All built-in names used in the program are now converted to "built-in name references" in a first step. Unsupported built-ins like e.g. zip, for which Nuitka has no own code or understanding yet, remained as "module variables", which made access to them slow, and difficult to recognize.

    • Added optimization for module attributes __file__, __doc__ and __package__ if they are read only. It's the same as __name__.

    • โž• Added optimization for slices and subscripts of "compile time constant" values. These will play a more important role, once value propagation makes them more frequent.

    Organizational

    • ๐Ÿš€ Created a "change log" from the previous release announcements. It's as ReStructured Text and converted to PDF for the release as well, but I chose not to include that in Debian, because it's so easy to generate the PDF on that yourself.

    • ๐Ÿš€ The posting of release announcements is now prepared by a script that converts the ReStructured Text to HTML and adds it to Wordpress as a draft posting or updates it, until it's release time. Simple, sweet and elegant.

    Cleanups

    • Split out the nuitka.nodes.Nodes module into many topic nodes, so that there are now nuitka.nodes.BoolNodes or nuitka.nodes.LoopNodes to host nodes of similar kinds, so that it is now cleaner.

    • Split del statements into their own node kind, and use much simpler node structures for them. The following blocks are absolutely the same:

    .. code-block:: python

     del a, b.c, d
    

    .. code-block:: python

     del a
     del b.c
     del d
    

    So that's now represented in the node tree. And even more complex looking cases, like this one, also the same:

    .. code-block:: python

     del a, (b.c, d)
    

    This one gives a different parse tree, but the same bytecode. And so Nuitka need no longer concern itself with this at all, and can remove the tuple from the parse tree immediately. That makes them easy to handle. As you may have noted already, it also means, there is no way to enforce that two things are deleted or none at all.

    • ๐Ÿ— Turned the function and class builder statements into mere assignment statements, where defaults and base classes are handled by wrapping expressions.

    Previously they are also kind of assignment statements too, which is not needed. Now they were reduced to only handle the bases for classes and the defaults for functions and make optional.

    • ๐Ÿ”จ Refactored the decorator handling to the tree building stage, presenting them as function calls on "function body expression" or class body expression".

    This allowed to remove the special code for decorators from code generation and C++ templates, making decorations easy subjects for future optimization, as they practically are now just function calls.

    .. code-block:: python

     @some_classdecorator
     class C:
         @staticmethod
         def f():
             pass
    

    It's just a different form of writing things. Nothing requires the implementation of decorators, it's just functions calls with function bodies before the assignment.

    The following is only similar:

    .. code-block:: python

     class C:
         def f():
             pass
    
         f = staticmethod(f)
    
     C = some_classdecorator(C)
    

    It's only similar, because the assignment to an intermediate value of C and f is not done, and if an exception was raised by the decoration, that name could persist. For Nuitka, the function and class body, before having a name, are an expression, and so can of course be passed to decorators already.

    • The in-place assignments statements are now handled using temporary variable blocks

    Adding support for scoped temporary variables and references to them, it was possible to re-formulate in-place assignments expressions as normal look-ups, in-place operation call and then assignment statement. This allowed to remove static templates and will yield even better generated code in the future.

    • The for loop used to have has a "source" expression as child, and the iterator over it was only taken at the code generation level, so that step was therefore invisible to optimization. Moved it to tree building stage instead, where optimization can work on it then.

    • ๐Ÿ— Tree building now generally allows statement sequences to be None everywhere, and pass statements are immediately eliminated from them immediately. Empty statement sequences are now forbidden to exist.

    • Moved the optimization for __name__ to compute node of variable references, where it doesn't need anything complex to replace with the constant value if it's only read.

    • โž• Added new bases classes and mix-in classes dedicated to expressions, giving a place for some defaults.

    • Made the built-in code more reusable.

    ๐Ÿ†• New Tests

    • โž• Added some more diagnostic tests about complex assignment and del statements.

    • โž• Added syntax test for star import on function level, that must fail on Python3.

    • โž• Added syntax test for duplicate argument name.

    • โž• Added syntax test for global on a function argument name.

    Summary

    ๐Ÿ— The decorator and building changes, the assignment changes, and the node cleanups are all very important progress for the type inference work, because ๐Ÿšš they remove special casing the that previously would have been required. ๐Ÿ— Lambdas and functions now really are the same thing right after tree building. The in-place assignments are now merely done using standard assignment code, the built functions and classes are now assigned to names in assignment statements, much more consistency there.

    Yet, even more work will be needed in the same direction. There may e.g. be work required to cover with statements as well. And assignments will become no more complex than unpacking from a temporary variable.

    ๐Ÿš€ For this release, there is only minimal progress on the Python3 front, despite ๐Ÿ‘ the syntax support, which is only minuscule progress. The remaining tasks appear all more or less difficult work that I don't want to touch now.

    ๐Ÿš€ There are still remaining steps, but we can foresee that a release may be done that finally actually does type inference and becomes the effective Python compiler this project is all about.

  • v0.3.19 Changes

    ๐Ÿ›  This time there are a few bug fixes, major cleanups, more Python3 support, and ๐Ÿš€ even new features. A lot things in this are justifying a new release.

    ๐Ÿ› Bug fixes

    • The man pages of nuitka and nuitka-python had no special layout for the option groups and broken whitespace for --recurse-to option. Also --g++-only was only partially bold. Released as 0.3.18.1 hot fix already.

    • ๐Ÿ The command line length improvement we made to Scons for Windows was not portable to Python2.6. Released as 0.3.18.2 hot fix already.

    • ๐Ÿ“ฆ Code to detect already considered packages detection was not portable to Windows, for one case, there was still a use of / instead of using a joinpath call. Released as 0.3.18.3 already.

    • ๐Ÿ‘€ A call to the range built-in with no arguments would crash the compiler, see Issue#29 <http://bugs.nuitka.net/issue29>__. Released as 0.3.18.4 already.

    • Compatibility Fix: When rich comparison operators returned false value other False, for comparison chains, these would not be used, but False instead, see .

    • The support for __import__ didn't cover keyword arguments, these were simply ignored. See Issue#28 <http://bugs.nuitka.net/issue28>__. Fixed, but no warning is given yet.

    ๐Ÿ†• New Features

    • A new option has been added, one can now specify --recurse-directory and Nuitka will attempt to embed these modules even if not obviously imported. This is not yet working perfect yet, but will receive future improvements.

    • โž• Added support for the exec built-in of Python3, this enables us to run one more basic test, GlobalStatement.py with Python3. The test ExecEval.py nearly works now.

    ๐Ÿ†• New Optimization

    • โšก๏ธ The no arguments range() call now optimized into the static CPython exception it raises.

    • โšก๏ธ Parts of comparison chains with constant arguments are now optimized away.

    Cleanups

    • Simplified the CPythonExpressionComparison node, it now always has only 2 operands.

    If there are more, the so called "comparison chain", it's done via and with assignments to temporary variables, which are expressed by a new node type CPythonExpressionTempVariableRef. This allowed to remove expression_temps from C++ code templates and generation, reducing the overall complexity.

    • When executing a module (--execute but not --exe), no longer does Nuitka import it into itself, instead a new interpreter is launched with a fresh environment.

    • The calls to the variadic MAKE_TUPLE were replaced with calls the MAKE_TUPLExx (where xx is the number of arguments), that are generated on a as-needed basis. This gives more readable code, because no EVAL_ORDERED_xx is needed at call site anymore.

    • ๐Ÿšš Many node classes have moved to new modules in nuitka.nodes and grouped by theme. That makes them more accessible.

    • ๐Ÿšš The choosing of the debug python has moved from Scons to Nuitka itself. That way it can respect the sys.abiflags and works with Python3.

    • The replacing of .py in filenames was made more robust. No longer is str.replace used, but instead proper means to assure that having .py as other parts of the filenames won't be a trouble.

    • Module recursion was changed into its own module, instead of being hidden in the optimization that considers import statements.

    • ๐Ÿ‘• As always, some PyLint work, and some minor TODO were solved.

    Organizational

    • โž• Added more information to the "Developer Manual <http://nuitka.net/doc/developer-manual.html>__", e.g. documenting the tree changes for assert to become a conditional statement with a raise statement, etc.

    • ๐Ÿ“ฆ The Debian package is as of this version verified to be installable and functional on to Ubuntu Natty, Maverick, Oneiric, and Precise.

    • โž• Added support to specify the binary under test with a NUITKA environment, so the test framework can run with installed version of Nuitka too.

    • ๐Ÿ Made sure the test runners work under Windows as well. Required making them more portable. And a workaround for os.execl not propagating exit codes under Windows. See Issue#26 <http://bugs.nuitka.net/issue26>__ for more information.

    • ๐Ÿ For windows target the MinGW library is now linked statically. That means there is no requirement for MinGW to be in the PATH or even installed to execute the binary.

    ๐Ÿ†• New Tests

    • The basic, programs, syntax, and reflected were made executable under Windows. Occasionally this meant to make the test runners more portable, or to work around limitations.

    • โž• Added test to cover return values of rich comparisons in comparison chains, and order of argument evaluation for comparison chains.

    • โœ… The Referencing.py test was made portable to Python3.

    • ๐Ÿ‘ป Cover no arguments range() exception as well.

    • โž• Added test to demonstrate that --recurse-directory actually works. This is using an __import__ that cannot be predicted at run time (yet).

    • ๐Ÿ“ฆ The created source package is now tested on pbuilder chroots to be pass installation and the basic tests, in addition to the full tests during package build time on these chroots. This will make sure, that Nuitka works fine on Ubuntu Natty and doesn't break without notice.

    Summary

    ๐Ÿš€ This releases contains many changes. The "temporary variable ref" and ๐Ÿ‘€ "assignment expression" work is ground breaking. I foresee that it will lead to even more simplifications of code generation in the future, when e.g. in-place assignments can be reduced to assignments to temporary variables and conditional statements.

    ๐Ÿ While there were many improvements related to Windows support and fixing ๐Ÿ“ฆ portability bugs, or the Debian package, the real focus is the optimization work, which will ultimately end with "value propagation" working.

    These are the real focus. The old comparison chain handling was a big wart. Working, but no way understood by any form of analysis in Nuitka. Now they have a structure which makes their code generation based on semantics and allows for ๐Ÿ‘€ future optimization to see through them.

    Going down this route is an important preparatory step. And there will be more work like this needed. Consider e.g. handling of in-place assignments. With an "assignment expression" to a "temporary variable ref", these become the same as ๐Ÿ‘‰ user code using such a variable. There will be more of these to find.

    ๐Ÿš€ So, that is where the focus is. The release now was mostly aiming at getting ๐Ÿ›  involved fixes out. The bug fixed by comparison chain reworking, and the __import__ related one, were not suitable for hot fix releases, so that is ๐Ÿš€ why the 0.3.19 release had to occur now. But with plugin support, with this ๐Ÿ‘ comparison chain cleanup, with improved Python3 support, and so on, there was plenty of good stuff already, also worth to get out.

  • v0.3.18 Changes

    ๐Ÿš€ This is to inform you about the new stable release of Nuitka. This time there ๐Ÿš€ are a few bug fixes, and the important step that triggered the release: Nuitka has entered Debian Unstable. So you if want, you will get stable Nuitka ๐Ÿš€ releases from now on via apt-get install nuitka.

    ๐Ÿš€ The release cycle was too short to have much focus. It merely includes fixes, ๐Ÿ›  which were available as hot fixes, and some additional optimization and node tree cleanups, as well as source cleanups. But not much else.

    ๐Ÿ› Bug fixes

    • โšก๏ธ Conditional statements with both branches empty were not optimized away in all cases, triggering an assertion of code generation. Issue#16 <http://bugs.nuitka.net/issue16>__. Released as 0.3.17a hot fix already.

    • ๐Ÿ“ฆ Nuitka was considering directories to contain packages that had no "init.py" which could lead to errors when it couldn't find the package later in the compilation process. Released as 0.3.17a hot fix already.

    • When providing locals() to exec statements, this was not making the locals() writable. The logic to detect the case that default value is used (None) and be pessimistic about it, didn't consider the actual value locals(). Released as 0.3.17b hot fix already.

    • 0๏ธโƒฃ Compatibility Fix: When no defaults are given, CPython uses None for func.func_defaults, but Nuitka had been using None.

    ๐Ÿ†• New Optimization

    • If the condition of assert statements can be predicted, these are now optimized in a static raise or removed.

    • For built-in name references, there is now dedicated code to look them up, that doesn't check the module level at all. Currently these are used in only a few cases though.

    • ๐Ÿ–จ Cleaner code is generated for the simple case of print statements. This is not only faster code, it's also more readable.

    Cleanups

    • โœ‚ Removed the CPythonStatementAssert node.

    It's not needed, instead at tree building, assert statements are converted to conditional statements with the asserted condition result inverted and a raise statement with AssertionError and the assertion argument.

    This allowed to remove code and complexity from the subsequent steps of Nuitka, and enabled existing optimization to work on assert statements as well.

    • ๐Ÿšš Moved built-in exception names and built-in names to a new module nuitka.Builtins instead of having in other places. This was previously a bit spread-out and misplaced.

    • โž• Added cumulative tags to node classes for use in checks. Use it annotate which node kinds to visit in e.g. per scope finalization steps. That avoids kinds and class checks.

    • ๐Ÿ†• New node for built-in name lookups

    This allowed to remove tricks played with adding module variable lookups for staticmethod when adding them for __new__ or module variable lookups for str when predicting the result of type('a'), which was unlikely to cause a problem, but an important TODO item still.

    Organizational

    • โšก๏ธ The "Download" <../pages/download.html>__ page is now finally updated for releases automatically.

    This closes Issue#7 <http://bugs.nuitka.net/issue7>__ completely. Up to this release, I had to manually edit that page, but now mastered the art of upload via XMLRCP and a Python script, so that don't loose as much time with editing, checking it, etc.

    • ๐Ÿ“ฆ The Debian package is backportable to Ubuntu Natty, Maverick, Oneiric, I expect to make a separate announcement with links to packages.

    • โœ… Made sure the test runners worth with bare python2.6 as well.

    ๐Ÿ†• New Tests

    • โž• Added some tests intended for type inference development.

    Summary

    ๐Ÿš€ This releases contains not as much changes as others, mostly because it's the intended base for a Debian upload.

    The exec fix was detected by continued work on the branch feature/minimize_CPython26_tests_diff branch, but that work is now complete.

    It is being made pretty (many git rebase iterations) with lots of Issues being โž• added to the bug tracker and referenced for each change. The intention is to have a clean commits repository with the changed made.

    But of course, the real excitement is the "type inference" work. It will give a huge boost to Nuitka. With this in place, new benchmarks may make sense. I am working on getting it off the ground, but also to make us more efficient.

    So when I learn something. e.g. assert is not special, I apply it to the develop branch immediately, to keep the differences as small as possible, and to immediately benefit from such improvements.

  • v0.3.17 Changes

    ๐Ÿš€ This is to inform you about the new stable release of Nuitka. This time there ๐Ÿ›  are a few bug fixes, lots of very important organisational work, and yet again ๐Ÿ‘Œ improved compatibility and cleanups. Also huge is the advance in making --deep go away and making the recursion of Nuitka controllable, which means ๐Ÿ“ฆ a lot for scalability of projects that use a lot of packages that use other ๐Ÿ“ฆ packages, because now you can choose which ones to embed and which ones one.

    ๐Ÿš€ The release cycle had a focus on improving the quality of the test scripts, the packaging, and generally to prepare the work on "type inference" in a new ๐Ÿ”‹ feature branch.

    I have also continued to work towards CPython3.2 compatibility, and this ๐Ÿ”– version, while not there, supports Python3 with a large subset of the basic โœ… tests programs running fine (of course via 2to3 conversion) without ๐Ÿ‘€ trouble. There is still work to do, exceptions don't seem to work fully yet, ๐Ÿ‘€ parameter parsing seems to have changed, etc. but it seems that CPython3.2 is going to work one day.

    And there has been a lot of effort, to address the Debian packaging to be cleaner and more complete, addressing issues that prevented it from entering the Debian repository.

    ๐Ÿ› Bug fixes

    • ๐Ÿ›  Fixed the handling of modules and packages of the same name, but with different casing. Problem showed under Windows only. Released as 0.3.16a hot fix already.

    • ๐Ÿ›  Fixed an error where the command line length of Windows was exceeded when many modules were embedded, Christopher Tott provided a fix for it. Released as 0.3.16a hot fix already.

    • ๐Ÿ›  Fix, avoid to introduce new variables for where built-in exception references are sufficient. Released as 0.3.16b hot fix already.

    • ๐Ÿ†• Fix, add the missing staticmethod decorator to __new__ methods before resolving the scopes of variables, this avoids the use of that variable before it was assigned a scope. Released as 0.3.16b hot fix already.

    ๐Ÿ†• New Features

    • โœจ Enhanced compatibility again, provide enough co_varnames in the code objects, so that slicing them up to code_object.co_argcount will work. They are needed by inspect module and might be used by some decorators as well.

    • ๐Ÿ†• New options to control the recursion:

    --recurse-none (do not warn about not-done recursions) --recurse-all (recurse to all otherwise warned modules) --recurse-to (confirm to recurse to those modules) --recurse-not-to (confirm to not recurse to those modules)

    ๐Ÿ†• New Optimization

    • The optimization of constant conditional expressions was not done yet. Added this missing constant propagation case.

    • Eliminate near empty statement sequences (only contain a pass statement) in more places, giving a cleaner node structure for many constructs.

    • ๐Ÿ‘‰ Use the pickle "protocol 2" on CPython2 except for unicode strings where it does not work well. It gives a more compressed and binary representation, that is generally more efficient to un-stream as well. Also use the cPickle protocol, the use of pickle was not really necessary anymore.

    Organizational

    • โž• Added a "Developer Manual <http://nuitka.net/doc/developer-manual.html>__" to the release. It's incomplete, but it details some of the existing stuff, coding rules, plans for "type inference", etc.

    • ๐Ÿ‘Œ Improved the --help output to use metavar where applicable. This makes it more readable for some options.

    • Instead of error message, give help output when no module or program file name was given. This makes Nuitka help out more convenient.

    • Consistently use #!/usr/bin/env python for all scripts, this was previously only done for some of them.

    • ๐Ÿ‘• Ported the PyLint check script to Python as well, enhancing it on the way to check the exit code, and to only output changes things, as well as making the output of warnings for TODO items optional.

    • ๐Ÿ‘• All scripts used for testing, PyLint checking, etc. now work with Python3 as well. Most useful on Arch Linux, where it's also already the default for Python.

    • ๐Ÿ’… The help output of Nuitka was polished a lot more. It is now more readable and uses option groups to combine related options together.

    • โœ… Make the tests run without any dependence on PATH to contain the executables of Nuitka. This makes it easier to use.

    • โž• Add license texts to 3rd party file that were missing them, apply licensecheck results to cleanup Nuitka. Also removed own copyright statement from in-line copy of Scons, it had been added by accident only.

    • ๐Ÿš€ Release the tests that I own as well as the Debian packaging I created under "Apache License 2.0" which is very liberal, meaning every project will be able to use it.

    • Don't require copyright assignment for contributions anymore, instead only "Apache License 2.0", the future Nuitka license, so that the code won't be a problem when changing the license of all of Nuitka to that license.

    • Give contributors listed in the "User Manual <http://nuitka.net/doc/user-manual.html>__" an exception to the GPL terms until Nuitka is licensed under "Apache License 2.0" as well.

    • โž• Added an --experimental option which can be used to control experimental features, like the one currently being added on branch feature/ctypes_annotation, where "type inference" is currently only activated when that option is given. For this stable release, it does nothing.

    • Check the static C++ files of Nuitka with cppcheck as well. Didn't find anything.

    • ๐Ÿง Arch Linux packages have been contributed, these are linked for download, but the stable package may lag behind a bit.

    Cleanups

    • ๐Ÿ”„ Changed not boolean operation to become a normal operator. Changed and and or boolean operators to a new base class, and making their interface more similar to that of operations.

    • โž• Added cumulative tags to node classes for use in checks. Use it annotate which node kinds to visit in e.g. per scope finalization steps. That avoids kinds and class checks.

    • โœจ Enhanced the "visitor" interface to provide more kinds of callbacks, enhanced the way "each scope" visiting is achieved by generalizing is as "child has not tag 'closure_taker'" and that for every "node that has tag 'closure_taker'".

    • ๐Ÿ“ฆ Moved SyntaxHighlighting module to nuitka.gui package where it belongs.

    • 0๏ธโƒฃ More white listing work for imports. As recursion is now the default, and leads to warnings for non-existent modules, the CPython tests gave a lot of good candidates for import errors that were white listed.

    • โœ… Consistently use nuitka in test scripts, as there isn't a Nuitka.py on all platforms. The later is scheduled for removal.

    • ๐Ÿ‘• Some more PyLint cleanups.

    ๐Ÿ†• New Tests

    • โœ… Make sure the basic tests pass with CPython or else fail the test. This is to prevent false positives, where a test passes, but only because it fails in CPython early on and then does so with Nuitka too. For the syntax tests we make sure they fail.

    • โœ… The basic tests can now be run with PYTHON=python3.2 and use 2to3 conversion in that case. Also the currently not passing tests are not run, so the passing tests continue to do so, with this run from the release test script check-release.

    • ๐Ÿš€ Include the syntax tests in release tests as well.

    • ๐Ÿ”„ Changed many existing tests so that they can run under CPython3 too. Of course this is via 2to3 conversion.

    • โœ… Don't fail if the CPython test suites are not there.

    Currently they remain largely unpublished, and as such are mostly only available to me (exception, feature/minimize_CPython26_tests_diff branch references the CPython2.6 tests repository, but that remains work in progress).

    • โœ… For the compile itself test: Make the presence of the Scons in-line copy optional, the Debian package doesn't contain it.

    • ๐Ÿ Also make it more portable, so it runs under Windows too, and allow to choose the Python version to test. Check this test with both CPython2.6 and CPython2.7 not only the default Python.

    • ๐Ÿ— Before releasing, test that the created Debian package builds fine in a minimal Debian unstable chroot, and passes all the tests included in the package (basics, syntax, programs, reflected). Also many other Debian packaging improvements.

    Summary

    ๐Ÿš€ The "git flow" was used again in this release cycle and proved to be useful not only for hot fix, but also for creating the branch feature/ctypes_annotation and rebasing it often while things are still flowing.

    ๐Ÿš€ The few hot fixes didn't require a new release, but the many organizational ๐Ÿ‘Œ improvements and the new features did warrant the new release, because of ๐Ÿš€ e.g. the much better test handling in this release and the improved recursion control.

    ๐Ÿ‘ The work on Python3 support has slowed down a bit. I mostly only added some bits for compatibility, but generally it has slowed down. I wanted to make sure it doesn't regress by accident, so running with CPython3.2 is now part of the ๐Ÿš€ normal release tests.

    What's still missing is more "hg" completeness. Only the co_varnames work for inspect was going in that direction, and this has slowed down. It was more important to make Nuitka's recursion more accessible with the new options, so that was done first.

    And of course, the real excitement is the "type inference" work. It will give a ๐Ÿ‘€ huge boost to Nuitka, and I am happy that it seems to go well. With this in place, new benchmarks may make sense. I am working on getting it off the ground, so other people can work on it too. My idea of ctypes native calls ๐Ÿ‘ may become true sooner than expected. To support that, I would like to add more tools to make sure we discover changes earlier on, checking the XML โœ… representations of tests to discover improvements and regressions more clearly.

  • v0.3.16 Changes

    ๐Ÿ›  This time there are many bug fixes, some important scalability work, and again ๐Ÿ‘Œ improved compatibility and cleanups.

    ๐Ÿš€ The release cycle had a focus on fixing the bug reports I received. I have also continued to look at CPython3 compatibility, and this is the first version to ๐Ÿ‘Œ support Python3 somewhat, at least some of the basic tests programs run (of course via 2to3 conversion) without trouble. I don't know when, but it ๐Ÿ‘€ seems that it's going to work one day.

    Also there has an effort to make the Debian packaging cleaner, addressing all kinds of small issues that prevented it from entering the Debian repository. It's still not there, but it's making progress.

    ๐Ÿ› Bug fixes

    • ๐Ÿ›  Fixed a packaging problem for Linux and x64 platform, the new swapFiber.S file for the fiber management was not included. Released as 0.3.15a hot fix already.

    • ๐Ÿ›  Fixed an error where optimization was performed on removed unreachable code, which lead to an error. Released as 0.3.15b hot fix already.

    • Fixed an issue with __import__ and recursion not happening in any case, because when it did, it failed due to not being ported to new internal APIs. Released as 0.3.15c hot fix already.

    • ๐Ÿ›  Fixed eval() and locals() to be supported in generator expressions and contractions too. Released as 0.3.15d hot fix already.

    • ๐Ÿ›  Fixed the Windows batch files nuitka.bat and nuitka-python.bat to not output the rem statements with the copyright header. Released as 0.3.15d hot fix already.

    • ๐Ÿ›  Fixed re-raise with raise, but without a current exception set. Released as 0.3.15e hot fix already.

    • ๐Ÿ›  Fixed vars() call on the module level, needs to be treated as globals(). Released as 0.3.15e hot fix already.

    • ๐Ÿ›  Fix handling of broken new lines in source files. Read the source code in "universal line ending mode". Released as 0.3.15f hot fix already.

    • Fixed handling of constant module attribute __name__ being replaced. Don't replace local variables of the same name too. Released as 0.3.15g hot fix already.

    • ๐Ÿ›  Fixed assigning to True, False or None. There was this old TODO, and some code has compatibility craft that does it. Released as 0.3.15g hot fix already.

    • ๐Ÿ›  Fix constant dictionaries not always being recognized as shared. Released as 0.3.15g hot fix already.

    • ๐Ÿ›  Fix generator function objects to not require a return frame to exist. In finalize cleanup it may not.

    • ๐Ÿ›  Fixed non-execution of cleanup codes that e.g. flush sys.stdout, by adding Py_Finalize().

    • ๐Ÿ›  Fix throw() method of generator expression objects to not check arguments properly.

    • ๐Ÿ›  Fix missing fallback to subscript operations for slicing with non-indexable objects.

    • ๐Ÿ›  Fix, in-place subscript operations could fail to apply the update, if the intermediate object was e.g. a list and the handle just not changed by the operation, but e.g. the length did.

    • ๐Ÿ›  Fix, the future spec was not properly preserving the future division flag.

    ๐Ÿ†• New Optimization

    • ๐Ÿ‘ The optimization scales now much better, because per-module optimization only require the module to be reconsidered, but not all modules all the time. With many modules recursed into, this makes a huge difference in compilation time.

    • โšก๏ธ The creation of dictionaries from constants is now also optimized.

    ๐Ÿ†• New Features

    • 0๏ธโƒฃ As a new feature functions now have the func_defaults and __defaults__ attribute. It works only well for non-nested parameters and is not yet fully integrated into the parameter parsing. This improves the compatibility somewhat already though.

    • The names True, False and None are now converted to constants only when they are read-only module variables.

    • The PYTHONPATH variable is now cleared when immediately executing a compiled binary unless --execute-with-pythonpath is given, in which case it is preserved. This allows to make sure that a binary is in fact containing everything required.

    Organizational

    • ๐Ÿ’… The help output of Nuitka was polished a lot more. It is now more readable and uses option groups to combine related options together.

    • ๐Ÿ‘• The in-line copy of Scons is not checked with PyLint anymore. We of course don't care.

    • โœ… Program tests are no longer executed in the program directory, so failed module inclusions become immediately obvious.

    • โœ… The basic tests can now be run with PYTHON=python3.2 and use 2to3 conversion in that case.

    Cleanups

    • ๐Ÿšš Moved tags to a separate module, make optimization emit only documented tags, checked against the list of allowed ones.

    • ๐Ÿ‘• The Debian package has seen lots of improvements, to make it "lintian clean", even in pedantic mode. The homepage of Nuitka is listed, a watch file can check for new releases, the git repository and the gitweb are referenced, etc.

    • ๐Ÿ Use os.path.join in more of the test code to achieve more Windows portability for them.

    • ๐Ÿ‘• Some more PyLint cleanups.

    ๐Ÿ†• New Tests

    • โœ… There is now a Crasher test, for tests that crashed Nuitka previously.

    • โž• Added a program test where the imported module does a sys.exit() and make sure it really doesn't continue after the SystemExit exception that creates.

    • Cover the type of __builtins__ in the main program and in imported modules in tests too. It's funny and differs between module and dict in CPython2.

    • โœ… Cover a final print statement without newline in the test. Must still receive a newline, which only happens when Py_Finalize() is called.

    • โž• Added test with functions that makes a raise without an exception set.

    • Cover the calling of vars() on module level too.

    • Cover the use of eval in contractions and generator expressions too.

    • 0๏ธโƒฃ Cover func_defaults and __default__ attributes for a function too.

    • โž• Added test function with two raise in an exception handler, so that one becomes dead code and removed without the crash.

    Summary

    ๐Ÿš€ The "git flow" was really great in this release cycle. There were many hot fix ๐Ÿš€ releases being made, so that the bugs could be addressed immediately without ๐Ÿš€ requiring the overhead of a full release. I believe that this makes Nuitka ๐Ÿ‘ clearly one of the best supported projects.

    This quick turn-around also encourages people to report more bugs, which is ๐Ÿ›  only good. And the structure is there to hold it. Of course, the many bug fixes meant that there is not as much new development, but that is not the priority, correctness is.

    The work on Python3 is a bit strange. I don't need Python3 at all. I also ๐Ÿšš believe it is that evil project to remove cruft from the Python core and make developers of all relevant Python software, add compatibility cruft to their software instead. Yet, I can't really stop to work on it. It has that appeal of small fixups here and there, and then something else works too.

    Python3 work is like when I was first struggling with Nuitka to pass the โœ… CPython2 unit tests for a first time. It's fun. And then it finds real actual ๐Ÿ› bugs that apply to CPython2 too. Not doing Py_Finalize (but having to), the slice operations shortcomings, the bug of subscript in-place, and so on. There ๐Ÿ‘ is likely more things hidden, and the earlier Python3 is supported, the more โœ… benefit from increased test covered.

    What's missing is more "hg" completeness. I think only the raise without 0๏ธโƒฃ exception set and the func_defaults issue were going into its direction, but it won't be enough yet.

  • v0.3.15 Changes

    ๐Ÿš€ This is to inform you about the new stable release of Nuitka. This time again ๐Ÿ›  many organizational improvements, some bug fixes, much improved compatibility and cleanups.

    ๐Ÿš€ This release cycle had a focus on packaging Nuitka for easier consumption, i.e. ๐Ÿ“š automatic packaging, making automatic uploads, improvement documentation, and generally cleaning things up, so that Nuitka becomes more compatible and โœ… ultimately capable to run the "hg" test suite. It's not there yet, but this is a huge jump for usability of Nuitka and its compatibility, again.

    ๐Ÿ‘ Then lots of changes that make Nuitka approach Python3 support, the generated ๐Ÿš€ C++ for at least one large example is compiling with this new release. It won't ๐Ÿ”— link, but there will be later releases.

    And there is a lot of cleanup going on, geared towards compatibility with line numbers in the frame object.

    ๐Ÿ› Bug fixes

    • The main module was using __main__ in tracebacks, but it must be <module>. Released as 0.3.14a hot fix already.

    • โ†ช Workaround for "execfile cannot be used as an expression". It wasn't possible to use execfile in an expression, only as a statement.

    But then there is crazy enough code in e.g. mercurial that uses it in a lambda function, which made the issue more prominent. The fix now allows it to be an expression, except on the class level, which wasn't seen yet.

    • ๐Ÿ The in-line copy of Scons was not complete enough to work for "Windows" or with --windows-target for cross compile. Fixed.

    • ๐Ÿš€ Cached frames didn't release the "back" frame, therefore holding variables of these longer than CPython does, which could cause ordering problems. Fixed for increased compatibility.

    • ๐Ÿ– Handle "yield outside of function" syntax error in compiled source correctly. This one was giving a Nuitka backtrace, now it gives a SyntaxError as it needs to.

    • Made syntax/indentation error output absolutely identical to CPython.

    • Using the frame objects f_lineno may fix endless amounts bugs related to traceback line numbers.

    ๐Ÿ†• New Features

    • ๐Ÿ Guesses the location of the MinGW compiler under Windows to default install location, so it need not be added to PATH environment variable. Removes the need to modify PATH environment just for Nuitka to find it.

    • โž• Added support for "lambda generators". You don't want to know what it is. Lets just say, it was the last absurd language feature out there, plus that didn't work. It now works perfect.

    Organizational

    • ๐Ÿ You can now download a Windows installer and a Debian package that works on Debian Testing, current Ubuntu and Mint Linux.

    • ๐Ÿ†• New release scripts give us the ability to have hot fix releases as download packages immediately. That means the "git flow" makes even more beneficial to the users.

    • Including the generated "README.pdf" in the distribution archives, so it can be read instead of "README.txt". The text file is fairly readable, due to the use of ReStructured Text, but the PDF is even nicer to read, due to e.g. syntax highlighting of the examples.

    • ๐Ÿ“‡ Renamed the main binaries to nuitka and nuitka-python, so that there is no dependency on case sensitive file systems.

    • ๐Ÿ For Windows there are batch files nuitka.bat and nuitka-python.bat to make Nuitka directly executable without finding the Python.exe, which the batch files can tell from their own location.

    • There are now man pages of nuitka and nuitka-python with examples for the most common use cases. They are of course included in the Debian package.

    • Don't strip the binary when executing it to analyse compiled binary with valgrind. It will give better information that way, without changing the code.

    ๐Ÿ†• New Optimization

    • Implemented swapcontext alike (swapFiber) for x64 to achieve 8 times speedup for Generators. It doesn't do useless syscalls to preserve signal masks. Now Nuitka is faster at frame switching than CPython on x64, which is already good by design.

    Cleanups

    • Using the frame objects to store current line of execution avoids the need to store it away in helper code at all. It ought to also help a lot with threading support, and makes Nuitka even more compatible, because now line numbers will be correct even outside tracebacks, but for mere stack frame dumps.

    • ๐Ÿ— Moved the for_return detection from code generation to tree building where it belongs. Yield statements used as return statements need slightly different code for Python2.6 difference. That solved an old TODO.

    • Much Python3 portability work. Sometimes even improving existing code, the Python compiler code had picked up a few points, where the latest Nuitka didn't work with Python3 anymore, when put to actual compile.

    The test covered only syntax, but e.g. meta classes need different code in CPython3, and that's now supported. Also helper code was made portable in more places, but not yet fully. This will need more work.

    • Cleaned up uses of debug defines, so they are now more consistent and in one place.

    • ๐Ÿ‘• Some more PyLint cleanups.

    ๐Ÿ†• New Tests

    • โœ… The tests are now executed by Python scripts and cover stderr output too. Before we only checked stdout. This unveiled a bunch of issues Nuitka had, but went unnoticed so far, and triggered e.g. the frame line number improvements.

    • โœ… Separate syntax tests.

    • โœ… The scripts to run the tests now are all in pure Python. This means, no more MinGW shell is needed to execute the tests.

    Summary

    โšก๏ธ The Debian package, Windows installer, etc. are now automatically updated and ๐Ÿš€ uploaded. From here on, there can be such packages for the hot fix releases too.

    ๐Ÿ‘ป The exception tracebacks are now correct by design, and better covered.

    ๐ŸŽ The generator performance work showed that the approach taken by Nuitka is in ๐Ÿ‘€ fact fast. It was fast on ARM already, but it's nice to see that it's now also fast on x64. Programs using generators will be affected a lot by this.

    ๐Ÿš€ Overall, this release brings Nuitka closer to usability. Better binary names, ๐Ÿ“š man pages, improved documentation, issue tracker, etc. all there now. I am in ๐Ÿ“ฆ fact now looking for a sponsor for the Debian package to upload it into Debian directly.

    โšก๏ธ .. admonition:: Update

    The upload to Debian happened for 0.3.18 and was done by Yaroslav Halchenko.

    ๐Ÿš€ What's missing is more "hg" completeness. The frame release issue helped it, but inspect.getargs() doesn't work yet, and is a topic for a future ๐Ÿš€ release. Won't be easy, as func_defaults will be an invasive change too.

  • v0.3.14 Changes

    ๐Ÿš€ This is to inform you about the new stable release of Nuitka. This time it ๐Ÿ›  contains mostly organisational improvements, some bug fixes, improved compatibility and cleanups.

    It is again the result of working towards compilation of a real program ๐Ÿ‘ (Mercurial). This time, I have added support for proper handling of compiled types by the inspect module.

    ๐Ÿ› Bug fixes

    • ๐Ÿ›  Fix for "Missing checks in parameter parsing with star list, star dict and positional arguments". There was whole in the checks for argument counts, now the correct error is given. Fixed in 0.3.13a already.

    • The simple slice operations with 2 values, not extended with 3 values, were not applying the correct order for evaluation. Fixed in 0.3.13a already.

    • The simple slice operations couldn't handle None as the value for lower or upper index. Fixed in 0.3.11a already.

    • The in-place simple slice operations evaluated the slice index expressions twice, which could cause problems if they had side effects. Fixed in 0.3.11a already.

    ๐Ÿ†• New Features

    • โš™ Run time patching the inspect module so it accepts compiled functions, compiled methods, and compiled generator objects. The test_inspect test of CPython is nearly working unchanged with this.

    • The generator functions didn't have CO_GENERATOR set in their code object, setting it made compatible with CPython in this regard too. The inspect module will therefore return correct value for inspect.isgeneratorfunction() too.

    ๐Ÿ†• New Optimization

    • Slice indexes that are None are now constant propagated as well.

    • Slightly more efficient code generation for dual star arg functions, removing useless checks.

    Cleanups

    • ๐Ÿ“ฆ Moved the Scons, static C++ files, and assembler files to new package nuitka.build where also now SconsInterface module lives.

    • ๐Ÿšš Moved the Qt dialog files to nuitka.gui

    • ๐Ÿšš Moved the "unfreezer" code to its own static C++ file.

    • ๐Ÿ‘• Some PyLint cleanups.

    ๐Ÿ†• New Tests

    • ๐Ÿ†• New test Recursion to cover recursive functions.

    • ๐Ÿ†• New test Inspection to cover the patching of inspect module.

    • โœ… Cover execfile on the class level as well in ExecEval test.

    • Cover evaluation order of simple slices in OrderCheck too.

    Organizational

    Please register and report issues you encounter with Nuitka. I have put all the known issues there and started to use it recently. It's Roundup based like http://bugs.python.org is, so people will find it familiar.

    • ๐Ÿš€ The setup.py is now apparently functional. The source releases for download are made it with, and it appears the binary distributions work too. We may now build a windows installer. It's currently in testing, we will make it available when finished.

    Summary

    The new source organisation makes packaging Nuitka really easy now. From here, ๐Ÿ we can likely provide "binary" package of Nuitka soon. A windows installer will be nice.

    The patching of inspect works wonders for compatibility for those programs that insist on checking types, instead of doing duck typing. The function call โœ… problem, was an issue found by the Mercurial test suite.

    โœ… For the "hg.exe" to pass all of its test suite, more work may be needed, this is the overall goal I am currently striving for. Once real world programs like Mercurial work, we can use these as more meaningful benchmarks and resume work on optimization.