Changelog History
Page 5
-
v0.5.9 Changes
๐ This release is mostly a maintenance release, bringing out minor compatibility ๐ improvements, and some standalone improvements. Also new options to control the recursion into modules are added.
๐ Bug Fixes
- Compatibility: Checks for iterators were using
PyIter_Check
which is buggy when running outside of Python core, because it's comparing pointers we don't see. Replaced withHAS_ITERNEXT
helper which compares against the pointer as extracting for a real non-iterator object.
.. code-block:: python
class Iterable: def __init__(self): self.consumed = 2 def __iter__(self): return Iterable() iter(Iterable()) # This is suppose to raise, but didn't with Nuitka
- Python3: Errors when creating class dictionaries raised by the
__prepare__
dictionary (e.g.enum
classes with wrong identifiers) were not immediately raised, but only by thetype
call.
This was not observable, but might have caused issues potentially.
๐ Standalone macOS: Shared libraries and extension modules didn't have their DLL load paths updated, but only the main binary. This is not sufficient for more complex programs.
๐ง Standalone Linux: Shared libraries copied into the
.dist
folder were read-only and executingchrpath
could potentially then fail. This has not been observed, but is a conclusion of macOS fix.Standalone: When freezing standard library, the path of Nuitka and the current directory remained in the search path, which could lead to looking at the wrong files.
Organizational
โก๏ธ The
getattr
built-in is now optimized for compile time constants if possible, even in the presence of adefault
argument. This is more a cleanup than actually useful yet.The calling of
PyCFunction
from normal Python extension modules got accelerated, especially for the no or single argument cases where Nuitka now avoids building the tuple.
๐ New Features
โ Added the option
--recurse-pattern
to include modules per filename, which for Python3 is the only way to not have them in a package automatically.โ Added the option
--generate-c++-only
to only generate the C++ source code without starting the compiler.
Mostly used for debugging and testing coverage. In the later case we do not want the C++ compiler to create any binary, but only to measure what would have been used.
Organizational
- ๐ Renamed the debug option
--c++-only
to--recompile-c++-only
to make its purpose more clear and there now is--generate-c++-only
too.
โ Tests
โ Added support for taking coverage of Nuitka in a test run on a given input file.
โ Added support for taking coverage for all Nuitka test runners, migrating them all to common code for searching.
โ Added uniform way of reporting skipped tests, not generally used yet.
Summary
๐ This release marks progress towards having coverage testing. Recent releases had made it clear that not all code of Nuitka is actually used at least once ๐ in our release tests. We aim at identifying these.
Another direction was to catch cases, where Nuitka leaks exceptions or is subject to leaked exceptions, which revealed previously unnoticed errors.
Important changes have been delayed, e.g. the closure variables will not yet ๐ use C++ objects to share storage, but proper
PyCellObject
for improved compatibility, and to approach a more "C-ish" status. These is unfinished code that does this. And the forward propagation of values is not enabled yet again either.๐ So this is an interim step to get the bug fixes and improvements accumulated ๐ out. Expect more actual changes in the next releases.
- Compatibility: Checks for iterators were using
-
v0.5.8 Changes
๐ This release has mainly a focus on cleanups and compatibility improvements. It ๐ also advances standalone support, and a few optimization improvements, but it ๐ mostly is a maintenance release, attacking long standing issues.
๐ Bug Fixes
- ๐ Compatibility Windows macOS: Fix importing on case insensitive systems.
It was not always working properly, if there was both a package
Something
andsomething
, by merit of having filesSomething/__init__.py
andsomething.py
.Standalone: The search path was preferring system directories and therefore could have conflicting DLLs.
Issue#144 <http://bugs.nuitka.net/issue144>
__.๐ Fix, the optimization of
getattr
with predictable result was crashing the compilation. This was a regression, fixed in 0.5.7.1 already.Compatibility: The name mangling inside classes also needs to be applied to global variables.
๐ Fix, proving
clang++
forCXX
was mistakingly thinking of it as ag++
and making version checks on it.Python3: Declaring
__class__
global is now aSyntaxError
before Python3.4.Standalone Python3: Making use of module state in extension modules was not working properly.
๐ New Features
- The filenames of source files as found in the
__file__
attribute are now made relative in standalone mode.
This should make it more apparent if things outside of the distribution folder are used, at the cost of tracebacks. Expect the default ability to copy the source code along in an upcoming release.
- โ Added experimental standalone mode support for PyQt5. At least headless mode should be working, plug-ins (needed for anything graphical) are not yet copied and will need more work.
Cleanup
No longer using
imp.find_module
anymore. To solve the casing issues we needed to make our own module finding implementation finally.๐ The name mangling was handled during code generation only. Moved to tree building instead.
More code generation cleanups. The compatible line numbers are now attached during tree building and therefore better preserved, as well as that code no longer polluting code generation as much.
Organizational
๐ฆ No more packages for openSUSE 12.1/12.2/12.3 and Fedora 17/18/19 as requested by the openSUSE Build Service.
โ Added RPM packages for Fedora 21 and CentOS 7 on openSUSE Build Service.
โ Tests
- โ Lots of test refinements for the CPython test suites to be run continuously in Buildbot for both Windows and Linux.
Summary
๐ This release brings about two major changes, each with the risk to break things.
One is that we finally started to have our own import logic, which has the risk to cause breakage, but apparently currently rather improved compatibility. The case issues were not fixable with standard library code.
The second one is that the
__file__
attributes for standalone mode is now no longer pointing to the original install and therefore will expose missing stuff sooner. This will have to be followed up with code to scan for missing "data" files later on.For SSA based optimization, there are cleanups in here, esp. the one removing ๐ the name mangling, allowing to remove special code for class variables. This ๐ makes the SSA tree more reliable. Hope is that the big step (forward ๐ propagation through variables) can be made in one of the next releases.
-
v0.5.7 Changes
๐ This release is brings a newly supported platform, bug fixes, and again lots of cleanups.
๐ Bug Fixes
- ๐ Fix, creation of dictionary and set literals with non-hashable indexes did not raise an exception.
.. code-block:: python
{[]: None} # This is now a TypeError
๐ New Optimization
- โก๏ธ Calls to the
dict
built-in with only keyword arguments are now optimized to mere dictionary creations. This is new for the case of non-constant arguments only of course.
.. code-block:: python
dict(a = b, c = d) # equivalent to {"a" : b, "c" : d}
โก๏ธ Slice
del
with indexable arguments are now using optimized code that avoids Python objects too. This was already done for slice look-ups.โ Added support for
bytearray
built-in.
Organizational
- โ Added support for OpenBSD with fiber implementation from library, as it has no context support.
Cleanups
- ๐ Moved slicing solutions for Python3 to the re-formulation stage. So far the slice nodes were used, but only at code generation time, there was made a distinction between Python2 and Python3 for them. Now these nodes are purely Python2 and slice objects are used universally for Python3.
โ Tests
โ The test runners now have common code to scan for the first file to compile, an implementation of the
search
mode. This will allow to introduce the ability to search for pattern matches, etc.โ More tests are directly executable with Python3.
โ Added
recurse_none
mode to test comparison, making using extra options for that purpose unnecessary.
Summary
This solves long standing issues with slicing and subscript not being properly ๐ distinguished in the Nuitka code. It also contains major bug fixes that really ๐ problematic. Due to the involved nature of these fixes they are made in this ๐ new release.
-
v0.5.6 Changes
๐ This release brings bug fixes, important new optimization, newly supported platforms, and important compatibility improvements. Progress on all fronts.
๐ Bug Fixes
Closure taking of global variables in member functions of classes that had a class variable of the same name was binding to the class variable as opposed to the module variable.
Overwriting compiled function's
__doc__
attribute more than once could corrupt the old value, leading to crashes.Issue#156 <http://bugs.nuitka.net/issue156>
__. Fixed in 0.5.5.2 already.Compatibility Python2: The
exec
statementexecfile
were changinglocals()
was given as an argument.
.. code-block:: python
def function(): a = 1 exec code in locals() # Cannot change local "a". exec code in None # Can change local "a" exec code
Previously Nuitka treated all 3 variants the same.
- Compatibility: Empty branches with a condition were reduced to only the condition, but they need in fact to also check the truth value:
.. code-block:: python
if condition: pass # must be treated as bool(condition) # and not (bug) condition
๐ Detection of Windows virtualenv was not working properly. Fixed in 0.5.5.2 already.
Large enough constants structures are now unstreamed via
marshal
module, avoiding large codes being generated with no point. Fixed in 0.5.5.2 already.๐ Windows: Pressing CTRL-C gave two stack traces, one from the re-execution of Nuitka which was rather pointless. Fixed in 0.5.5.1 already.
๐ Windows: Searching for virtualenv environments didn't terminate in all cases. Fixed in 0.5.5.1 already.
During installation from PyPI with Python3 versions, there were errors given for the Python2 only scons files.
Issue#153 <http://bugs.nuitka.net/issue153>
__. Fixed in 0.5.5.3 already.๐ Fix, the arguments of
yield from
expressions could be leaked.๐ Fix, closure taking of a class variable could have in a sub class where the module variable was meant.
.. code-block:: python
var = 1 class C: var = 2 class D: def f(): # was C.var, now correctly addressed top level var return var
๐ Fix, setting
CXX
environment variable because the installed gcc has too low version, wasn't affecting the version check at all.๐ Fix, on Debian/Ubuntu with
hardening-wrapper
installed the version check was always failing, because these report a shortened version number to Scons.
๐ New Optimization
Local variables that must be assigned also have no side effects, making use of SSA. This allows for a host of optimization to be applied to them as well, often yielding simpler access/assign code, and discovering in more cases that frames are not necessary.
Micro optimization to
dict
built-in for simpler code generation.
Organizational
โ Added support for ARM "hard float" architecture.
โ Added package for Ubuntu 14.10 for download.
โ Added package for openSUSE 13.2 for download.
Donations were used to buy a Cubox-i4 Pro. It got Debian Jessie installed on it, and will be used to run an even larger amount of tests.
๐ Made it more clear in the user documentation that the
.exe
suffix is used for all platforms, and why.โก๏ธ Generally updated information in user manual and developer manual about the optimization status.
Using Nikola 7.1 with external filters instead of our own, outdated branch for the web site.
Cleanups
๐ PyLint clean for the first time ever. We now have a Buildbot driven test that this stays that way.
Massive indentation cleanup of keyword argument calls. We have a rule to align the keywords, but as this was done manually, it could easily get out of touch. Now with a "autoformat" tool based on RedBaron, it's correct. Also, spacing around arguments is now automatically corrected. More to come.
For
exec
statements, the coping back to local variables is now an explicit node in the tree, leader to cleaner code generation, as it now uses normal variable assignment code generation.The
MaybeLocalVariables
became explicit about which variable they might be, and contribute to its SSA trace as well, which was incomplete before.โ Removed some cases of code duplication that were marked as TODO items. This often resulted in cleanups.
Do not use
replaceWith
on child nodes, that potentially were re-used during their computation.
Summary
๐ The release is mainly the result of consolidation work. While the previous ๐ release contained many important enhancements, this is another important step towards full SSA, closing one loop whole (class variables and
exec
functions), as well as applying it to local variables, largely extending its ๐ use.The amount of cleanups is tremendous, in huge part due to infrastructure ๐ problems that prevented release repeatedly. This reduces the technological debt very much.
More importantly, it would appear that now eliminating local and temporary variables that are not necessary is only a small step away. But as usual, while this may be easy to implement now, it will uncover more bugs in existing code, that we need to address before we continue.
-
v0.5.5 Changes
๐ This release is finally making full use of SSA analysis knowledge for code ๐ generation, leading to many enhancements over previous releases.
๐ It also adds support for Python3.4, which has been longer in the making, due to many rather subtle issues. In fact, even more work will be needed to fully solve remaining minor issues, but these should affect no real code.
๐ And then there is much improved support for using standalone mode together ๐ with virtualenv. This combination was not previously supported, but should work now.
๐ New Features
- โ Added support for Python3.4
This means support for
clear
method of frames to close generators, dynamic__qualname__
, affected byglobal
statements, tuples asyield from
arguments, improved error messages, additional checks, and many more detail changes.๐ New Optimization
- Using SSA knowledge, local variable assignments now no longer need to check if they need to release previous values, they know definitely for the most cases.
.. code-block:: python
def f(): a = 1 # This used to check if old value of "a" needs a release ...
- Using SSA knowledge, local variable references now no longer need to check for raising exceptions, let alone produce exceptions for cases, where that cannot be.
.. code-block:: python
def f(): a = 1 return a # This used to check if "a" is assigned
Using SSA knowledge, local variable references now are known if they can raise the
UnboundLocalError
exception or not. This allows to eliminate frame usages for many cases. Including the above example.Using less memory for keeping variable information.
Also using less memory for constant nodes.
๐ Bug Fixes
The standalone freezing code was reading Python source as UTF-8 and not using the code that handles the Python encoding properly. On some platforms there are files in standard library that are not encoded like that.
๐ง The fiber implementation for Linux amd64 was not working with glibc from RHEL 5. Fixed to use now multiple
int
to pass pointers as necessary. Also useuintptr_t
instead ofintprt_t
to transport pointers, which may be more optimal.Line numbers for exceptions were corrupted by
with
statements due to setting line numbers even for statements marked as internal.๐ Partial support for
win32com
by adding support for its hidden__path__
change.Python3: Finally figured out proper chaining of exceptions, given proper context messages for exception raised during the handling of exceptions.
Corrected C++ memory leak for each closure variable taken, each time a function object was created.
Python3: Raising exceptions with tracebacks already attached, wasn't using always them, but producing new ones instead.
Some constants could cause errors, as they cannot be handled with the
marshal
module as expected, e.g.(int,)
.Standalone: Make sure to propagate
sys.path
to the Python instance used to check for standard library import dependencies. This is important for virtualenv environments, which needsite.py
to set the path, which is not executed in that mode.๐ Windows: Added support for different path layout there, so using virtualenv should work there too.
โก๏ธ The code object flag "optimized" (fast locals as opposed to locals dictionary) for functions was set wrongly to value for the parent, but for frames inside it, one with the correct value. This lead to more code objects than necessary and false
co_flags
values attached to the function.Options passed to
nuitka-python
could get lost.
.. code-block:: sh
nuitka-python program.py argument1 argument2 ...
The above is supposed to compile program.py, execute it immediately and pass the arguments to it. But when Nuitka decides to restart itself, it would forget these options. It does so to e.g. disable hash randomization as it would affect code generation.
- ๐ป Raising tuples exception as exceptions was not compatible (Python2) or reference leaking (Python3).
โ Tests
โ Running
2to3
is now avoided for tests that are already running on both Python2 and Python3.โ Made XML based optimization tests work with Python3 too. Previously these were only working on Python2.
โ Added support for ignoring messages that come from linking against self compiled Pythons.
โ Added test case for threaded generators that tortures the fiber layer a bit and exposed issues on RHEL 5.
โ Made reference count test of compiled functions generic. No more code duplication, and automatic detection of shared stuff. Also a more clear interface for disabling test cases.
โ Added Python2 specific reference counting tests, so the other cases can be executed with Python3 directly, making debugging them less tedious.
Cleanups
Really important removal of "variable references". They didn't solve any problem anymore, but their complexity was not helpful either. This allowed to make SSA usable finally, and removed a lot of code.
โ Removed special code generation for parameter variables, and their dedicated classes, no more needed, as every variable access code is now optimized like this.
Stop using C++ class methods at all. Now only the destructor of local variables is actually supposed to do anything, and their are no methods anymore. The unused
var_name
got removed,setVariableValue
is now done manually.๐ Moved assertions for the fiber layer to a common place in the header, so they are executed on all platforms in debug mode.
๐ As usual, also a bunch of cleanups for PyLint were applied.
The
locals
built-in code now uses code generation for accessing local variable values instead having its own stuff.
Organizational
๐ The Python version 3.4 is now officially supported. There are a few problems open, that will be addressed in future releases, none of which will affect normal people though.
Major cleanup of Nuitka options.
- Windows specific stuff is now in a dedicated option group. This includes options for icon, disabling console, etc.
- There is now a dedicated group for controlling backend compiler choices and options.
Also pickup
g++44
automatically, which makes using Nuitka on CentOS5 more automatic.
Summary
๐ This release represents a very important step ahead. Using SSA for real stuff ๐ will allow us to build the trust necessary to take the next steps. Using the SSA information, we could start implementing more optimizations.
-
v0.5.4 Changes
๐ This release is aiming at preparatory changes to enable optimization based on SSA analysis, introducing a variable registry, so that variables no longer trace their references to themselves.
๐ Otherwise, MinGW64 support has been added, and lots of bug fixes were made to ๐ improve the compatibility.
๐ New Optimization
Using new variable registry, now properly detecting actual need for sharing variables. Optimization may discover that it is unnecessary to share a variable, and then it no longer is. This also allows
--debug
without it reporting unused variable warnings on Python3.Scons startup has been accelerated, removing scans for unused tools, and avoiding making more than one gcc version check.
๐ Bug Fixes
Compatibility: In case of unknown encodings, Nuitka was not giving the name of the problematic encoding in the error message. Fixed in 0.5.3.3 already.
Submodules with the same name as built-in modules were wrongly shadowed. Fixed in 0.5.3.2 already.
๐ฆ Python3: Added implementations of
is_package
to the meta path based loader.Python3.4: Added
find_spec
implementation to the meta path based loader for increased compatibility.Python3: Corrections for
--debug
to work with Python3 and MSVC compiler more often.๐ Fixed crash with
--show-scons
when no compiler was found. Fixed in 0.5.3.5 already.Standalone: Need to blacklist
lib2to3
from standard library as well. Fixed in 0.5.3.4 already.๐ Python3: Adapted to changes in
SyntaxError
on newer Python releases, there is now amsg
that can overridereason
.๐ Standalone Windows: Preserve
sys.executable
as it might be used to fork binaries.๐ Windows: The caching of Scons was not arch specific, and files could be used again, even if changing the arch from `
x86
tox86_64
or back.๐ Windows: On 32 bit Python it can happen that with large number of generators running concurrently (>1500), one cannot be started anymore. Raising an
MemoryError
now.
Organizational
โ Added support for MinGW64. Currently needs to be run with
PATH
environment properly set up.โก๏ธ Updated internal version of Scons to 2.3.2, which breaks support for VS 2008, but adds support for VS 2013 and VS 2012. The VS 2013 is now the recommended compiler.
โ Added RPM package and repository for RHEL 7.
The output of
--show-scons
now includes the used compiler, including the MSVC version.โ Added option
--msvc
to select the MSVC compiler version to use, which overrides automatic selection of the latest.โ Added option
-python-flag=no_warnings
to disable user and deprecation warnings at run time.๐ Repository for Ubuntu Raring was removed, no more supported by Ubuntu.
Cleanups
Made technical and logical sharing decisions separate functions and implement them in a dedicated variable registry.
๐ The Scons file has seen a major cleanup.
Summary
๐ This release is mostly a maintenance release. The Scons integrations has been heavily visited, as has been Python3 and esp. Python3.4 compatibility, and โ results from the now possible debug test runs.
Standalone should be even more practical now, and MinGW64 is an option for those cases, where MSVC is too slow.
-
v0.5.3 Changes
๐ This release is mostly a follow up, resolving points that have become possible to resolve after completing the C-ish evolution of Nuitka. So this is more of a ๐ service release.
๐ New Features
๐ Improved mode
--improved
now sets error lines more properly than CPython does in many cases.The
-python-flag=-S
mode now preservesPYTHONPATH
and therefore became usable with virtualenv.
๐ New Optimization
๐ป Line numbers of frames no longer get set unless an exception occurs, speeding up the normal path of execution.
For standalone mode, using
--python-flag-S
is now always possible and yields less module usage, resulting in smaller binaries and faster compilation.
๐ Bug Fixes
โก๏ธ Corrected an issue for frames being optimized away where in fact they are still necessary.
Issue#140 <http://bugs.nuitka.net/issue140>
__. Fixed in 0.5.2.1 already.๐ Fixed handling of exception tests as side effects. These could be remainders of optimization, but didn't have code generation. Fixed in 0.5.2.1 already.
Previously Nuitka only ever used the statement line as the line number for all the expression, even if it spawned multiple lines. Usually nothing important, and often even more correct, but sometimes not. Now the line number is most often the same as CPython in full compatibility mode, or better, see above.
Issue#9 <http://bugs.nuitka.net/issue9>
__.๐ Python3.4: Standalone mode for Windows is working now.
Standalone: Undo changes to
PYTHONPATH
orPYTHONHOME
allowing potentially forked CPython programs to run properly.๐ Standalone: Fixed import error when using PyQt and Python3.
๐ New Tests
โ For our testing approach, the improved line number handling means we can undo lots of changes that are no more necessary.
โ The compile library test has been extended to cover a third potential location where modules may live, covering the
matplotlib
module as a result.
Cleanups
- In Python2, the list contractions used to be re-formulated to be function calls that have no frame stack entry of their own right. This required some special handling, in e.g. closure taking, and determining variable sharing across functions.
This now got cleaned up to be properly in-lined in a
try
/finally
expression.The line number handling got simplified by pushing it into error exits only, removing the need to micro manage a line number stack which got removed.
๐ Use
intptr_t
overunsigned long
to store fiber code pointers, increasing portability.
Organizational
Providing own Debian/Ubuntu repositories for all relevant distributions.
๐ Windows MSI files for Python 3.4 were added.
๐ Hosting of the web site was moved to metal server with more RAM and performance.
Summary
๐ This release brings about structural simplification that is both a follow-up to ๐ C-ish, as well as results from a failed attempt to remove static "variable references" and be fully SSA based. It incorporates changes aimed at making this next step in Nuitka evolution smaller.
-
v0.5.2 Changes
๐ This is a major release, with huge changes to code generation that improve ๐ performance in a significant way. It is a the result of a long development period, and therefore contains a huge jump ahead.
๐ New Features
โ Added experimental support for Python 3.4, which is still work in progress.
โ Added support for virtualenv on macOS.
โ Added support for virtualenv on Windows.
โ Added support for macOS X standalone mode.
The code generation uses no header files anymore, therefore adding a module doesn't invalidate all compiled object files from caches anymore.
Constants code creation is now distributed, and constants referenced in a module are declared locally. This means that changing a module doesn't affect the validity of other modules object files from caches anymore.
๐ New Optimization
C-ish code generation uses less C++ classes and generates more C-like code. Explicit temporary objects are now used for statement temporary variables.
The constants creation code is no more in a single file, but distributed across all modules, with only shared values created in a single file. This means improved scalability. There are remaining bad modules, but more often, standalone mode is now fast.
๐ป Exception handling no longer uses C++ exception, therefore has become much faster.
Loops that only break are eliminated.
๐ Dead code after loops that do not break is now removed.
The
try
/finally
andtry
/except
constructs are now eliminated, where that is possible.๐จ The
try
/finally
part of the re-formulation forprint
statements is now only done when printing to a file, avoiding useless node tree bloat.Tuples and lists are now generated with faster code.
Locals and global variables are now access with more direct code.
โ Added support for the anonymous
code
type built-in.โ Added support for
compile
built-in.Generators that statically return immediately, e.g. due to optimization results, are no longer using frame objects.
The complex call helpers use no pseudo frames anymore. Previous code generation required to have them, but with C-ish code generation that is no more necessary, speeding up those kind of calls.
Modules with only code that cannot raise, need not have a frame created for them. This avoids useless code size bloat because of them. Previously the frame stack entry was mandatory.
๐ Bug Fixes
๐ Windows: The resource files were cached by Scons and re-used, even if the input changed. The could lead to corrupted incremental builds.
Issue#129 <http://bugs.nuitka.net/issue129>
__. Fixed in 0.5.1.1 already.๐ Windows: For functions with too many local variables, the MSVC failed with an error "C1026: parser stack overflow, program too complex". The rewritten code generation doesn't burden the compiler as much.
Issue#127 <http://bugs.nuitka.net/issue127>
__.Compatibility: The timing deletion of nested call arguments was different from C++. This shortcoming has been addressed in the rewritten code generation.
Issue#62 <http://bugs.nuitka.net/issue62>
__.Compatibility: The
__future__
flags andCO_FREECELL
were not present in frame flags. These were then not always properly inherited toeval
andexec
in all cases.Compatibility: Compiled frames for Python3 had
f_restricted
attribute, which is Python2 only. Removed it.Compatibility: The
SyntaxError
of having acontinue
in a finally clause is now properly raised.Python2: The
exec
statement with no locals argument provided, was preventing list contractions to take closure variables.Python2: Having the ASCII encoding declared in a module wasn't working.
Standalone: Included the
idna
encoding as well.Issue#135 <http://bugs.nuitka.net/issue135>
__.Standalone: For virtualenv, the file
orig-prefix.txt
needs to be present, now it's copied into the "dist" directory as well.Issue#126 <http://bugs.nuitka.net/issue126>
__. Fixed in 0.5.1.1 already.๐ Windows: Handle cases, where Python and user program are installed on different volumes.
Compatibility: Can now finally use
execfile
as an expression.Issue#5 <http://bugs.nuitka.net/issue5>
__ is finally fixed after all this time thanks to C-ish code generation.Compatibility: The order or call arguments deletion is now finally compatible.
Issue#62 <http://bugs.nuitka.net/issue62>
__ also is finally fixed. This too is thanks to C-ish code generation.Compatibility: Code object flags are now more compatible for Python3.
Standalone: Removing "rpath" settings of shared libraries and extension modules included. This makes standalone binaries more robust on Fedora 20.
Python2: Wasn't falsely rejecting
unicode
strings as values forint
andlong
variants with base argument provided.๐ Windows: For Python3.2 and 64 bits, global variable accesses could give false
NameError
exceptions. Fixed in 0.5.1.6 already.Compatibility: Many
exec
andeval
details have become more correctly, the argument handling is more compatible, and e.g. future flags are now passed along properly.Compatibility: Using
open
with no arguments is now giving the same error.
Organizational
Replying to email from the
issue tracker <http://bugs.nuitka.net>
__ works now.โ Added option name alias
--xml
for--dump-xml
.โ Added option name alias
--python-dbg
for--python-debug
, which actually might make it a bit more clear that it is about using the CPython debug run time.โ Remove option
--dump-tree
, it had been broken for a long time and unused in favor of XML dumps.๐ New digital art folder with 3D version of Nuitka logo. Thanks to Juan Carlos for creating it.
๐ Using "README.rst" instead of "README.txt" to make it look better on web pages.
More complete whitelisting of missing imports in standard library. These should give no warnings anymore.
โก๏ธ Updated the Nuitka GUI to the latest version, with enhanced features.
๐ The builds of releases and update of the
downloads page <http://nuitka.net/pages/download.html>
__ is now driven by Buildbot. Page will be automatically updated as updated binaries arrive.
Cleanups
Temporary keeper variables and the nodes to handle them are now unified with normal temporary variables, greatly simplifying variable handling on that level.
Less code is coming from templates, more is actually derived from the node tree instead.
Releasing the references to temporary variables is now always explicit in the node tree.
The publishing and preservation of exceptions in frames was turned into explicit nodes.
๐ป Exception handling is now done with a single handle that checks with branches on the exception. This eliminates exception handler nodes.
The
dir
built-in with no arguments is now re-formulated tolocals
orglobals
with their.keys()
attribute taken.Dramatic amounts of cleanups to code generation specialties, that got done right for the new C-ish code generation.
๐ New Tests
โ Warnings from MSVC are now error exits for
--debug
mode too, expanding the coverage of these tests.The outputs with
python-dbg
can now also be compared, allowing to expand test coverage for reference counts.โ Many of the basic tests are now executable with Python3 directly. This allows for easier debug.
โ The library compilation test is now also executed with Python3.
Summary
๐ This release would deserve more than a minor number increase. The C-ish code generation, is a huge body of work. In many ways, it lays ground to taking benefit of SSA results, that previously would not have been possible. In other ways, it's incomplete in not yet taking full advantage yet.
๐ The release contains so many improvements, that are not yet fully realized, but as a compiler, it also reflects a stable and improved state.
The important changes are about making SSA even more viable. Many of the ๐ป problematic cases, e.g. exception handlers, have been stream lined. A whole class of variables, temporary keepers, has been eliminated. This is big news in this domain.
For the standalone users, there are lots of refinements. There is esp. a lot of work to create code that doesn't show scalability issues. While some remain, the most important problems have been dealt with. Others are still in the pipeline.
More work will be needed to take full advantage. This has been explained in a
separate post <http://nuitka.net/posts/state-of-nuitka.html>
__ in greater detail. -
v0.5.1 Changes
๐ This release brings corrections and major improvements to how standalone mode performs. Much of it was contributed via patches and bug reports.
๐ Bug Fixes
๐ There was a crash when using
next
on a non-iterable. Fixed in 0.5.0.1 already.Module names with special characters not allowed in C identifiers were not fully supported.
Issue#118 <http://bugs.nuitka.net/issue118>
__. Fixed in 0.5.0.1 already.Name mangling for classes with leading underscores was not removing them from resulting attribute names. This broke at
__slots__
with private attributes for such classes.Issue#119 <http://bugs.nuitka.net/issue119>
__. Fixed in 0.5.0.1 already.๐ Standalone on Windows might need "cp430" encoding.
Issue#120 <http://bugs.nuitka.net/issue120>
__. Fixed in 0.5.0.2 already.Standalone mode didn't work with
lxml.etree
due to lack of hard coded dependencies. When a shared library imports things, Nuitka cannot detect it easily.๐ Wasn't working on macOS 64 bits due to using Linux 64 bits specific code.
Issue#123 <http://bugs.nuitka.net/issue123>
__. Fixed in 0.5.0.2 already.On MinGW the constants blob was not properly linked on some installations, this is now done differently (see below).
๐ New Features
- Memory usages are now traced with
--show-progress
allowing us to trace where things go wrong.
๐ New Optimization
0๏ธโฃ Standalone mode now includes standard library as bytecode by default. This is workaround scalability issues with many constants from many modules. Future releases are going to undo it.
๐ On Windows the constants blob is now stored as a resource, avoiding compilation via C code for MSVC as well. MinGW was changed to use the same code.
๐ New Tests
- โ Expanded test coverage for "standalone mode" demonstrating usage of "hex" encoding, PySide, and PyGtk packages.
Summary
๐ This release is mostly an interim maintenance release for standalone. Major ๐ changes that provide optimization beyond that, termed "C-ish code generation" ๐ are delayed for future releases.
๐ This release makes standalone practical which is an important point. Instead of hour long compilation, even for small programs, we are down to less than a minute.
The solution of the scalability issues with many constants from many modules will be top priority going forward. Since they are about how even single use constants are created all in one place, this will be easy, but as large changes are happening in "C-ish code generation", we are waiting for these to complete.
-
v0.5.0 Changes
๐ This release breaks interface compatibility, therefore the major version number ๐ change. Also "standalone mode" has seen significant improvements on both ๐ง Windows, and Linux. Should work much better now.
But consider that this part of Nuitka is still in its infancy. As it is not the top priority of mine for Nuitka, which primarily is intended as an super compatible accelerator of Python, it will continue to evolve nearby.
There is also many new optimization based on structural improvements in the direction of actual SSA.
๐ Bug Fixes
The "standalone mode" was not working on all Redhat, Fedora, and openSUSE platforms and gave warnings with older compilers. Fixed in 0.4.7.1 already.
The "standalone mode" was not including all useful encodings.
Issue#116 <http://bugs.nuitka.net/issue116>
__. Fixed in 0.4.7.2 already.0๏ธโฃ The "standalone mode" was defaulting to
--python-flag=-S
which disables the parsing of "site" module. That unfortunately made it necessary to reach some modules without modifyingPYTHONPATH
which conflicts with the "out-of-the-box" experience.๐ฆ The "standalone mode" is now handling packages properly and generally working on Windows as well.
The syntax error of having an all catching except clause and then a more specific one wasn't causing a
SyntaxError
with Nuitka.
.. code-block:: python
try: something() except: somehandling(): except TypeError: notallowed()
- A corruption bug was identified, when re-raising exceptions, the top entry of
the traceback was modified after usage. Depending on
malloc
this was potentially causing an endless loop when using it for output.
๐ New Features
- ๐ Windows: The "standalone" mode now properly detects used DLLs using
Dependency Walker <http://www.dependencywalker.com/>
__ which it offers to download and extra for you.
It is used as a replacement to
ldd
on Linux when building the binary, and as a replacement ofstrace
on Linux when running the tests to check that nothing is loaded from the outside.๐ New Optimization
- When iterating over
list
,set
, this is now automatically lowered totuples
avoiding the mutable container types.
So the following code is now equivalent:
.. code-block:: python
for x in [ a, b, c ]: ... # same as for x in (a, b, c): ...
For constants, this is even more effective, because for mutable constants, no more is it necessary to make a copy.
Python2: The iteration of large
range
is now automatically lowered toxrange
which is faster to loop over, and more memory efficient.โ Added support for the
xrange
built-in.The statement only expression optimization got generalized and now is capable of removing useless parts of operations, not only the whole thing when it has not side effects.
.. code-block:: python
[a,b] # same as a b
This works for all container types.
Another example is
type
built-in operation with single argument. When the result is not used, it need not be called... code-block:: python
type(a) # same as a
And another example
is
andis not
have no effect of their own as well, therefore:.. code-block:: python
a is b # same as a b
- โ Added proper handling of conditional expression branches in SSA based optimization. So far these branches were ignored, which only acceptable for temporary variables as created by tree building, but not other variable types. This is preparatory for introducing SSA for local variables.
Organizational
0๏ธโฃ The option
--exe
is now ignored and creating an executable is the default behavior ofnuitka
, a new option--module
allows to produce extension modules.๐ The binary
nuitka-python
was removed, and is replaced bynuitka-run
with now only implies--execute
on top of whatnuitka
is.๐ Using dedicated
Buildbot <http://buildbot.net>
__ for continuous integration testing and release creation as well.The
Downloads <http://nuitka.net/pages/download.html>
__ now offers MSI files for Win64 as well.๐ Discontinued the support for cross compilation to Win32. That was too limited and the design choice is to have a running CPython instance of matching architecture at Nuitka compile time.
๐ New Tests
- โ Expanded test coverage for "standalone mode" demonstrating usage of "hex" encoding, and PySide package.
Summary
0๏ธโฃ The "executable by default" interface change improves on the already high ease of use. The new optimization do not give all that much in terms of numbers, but are all signs of structural improvements, and it is steadily approaching the point, where the really interesting stuff will happen.
The progress for standalone mode is of course significant. It is still not quite there yet, but it is making quick progress now. This will attract a lot of attention hopefully.
๐ป As for optimization, the focus for it has shifted to making exception handlers 0๏ธโฃ work optimal by default (publish the exception to sys.exc_info() and create traceback only when necessary) and be based on standard branches. Removing ๐ special handling of exception handlers, will be the next big step. This release ๐ includes some correctness fixes stemming from that work already.