Currently the host build of Python 2 defaults to narrow unicode (UCS2),
ignoring the BR2_PACKAGE_PYTHON_UCS4 configuration option which may be
set to wide (UCS4).
This results in host and target Python packages which are incompatible
in subtle ways.
For example, installing wheels into the target fails when they are made
with the host python, citing incompatibility (as can be seen by the
package tags which may be "cp27u-manylinux1" instead of
"cp27mu-manylinux1").
Compiling the host Python 2 with the same UCS configuration as the
target ensures that the packages are compatible (and the tags match).
This does not affect Python 3 as support for narrow unicode was
deprecated in version 3.3, see https://www.python.org/dev/peps/pep-0393/
Thanks to Tony Breeds <tony@bakeyournoodle.com> for reporting this.
Signed-off-by: Chris Smart <mail@csmart.io>
[Thomas: add comment in the code.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Python is not able to detect if compiler double representation is
compliant with IEE754:
checking whether C doubles are little-endian IEEE 754 binary64... no
checking whether C doubles are big-endian IEEE 754 binary64... no
checking whether C doubles are ARM mixed-endian IEEE 754 binary64... no
Accordingly 'legacy' mode isused. It is possible to check this at
runtime by check if 'sys.float_repr_style' contains 'short' or
'legacy'. Calculus correctness is not garanteed with 'legacy'.
Problem is better described here:
http://stackoverflow.com/questions/29920294/what-causes-pythons-float-repr-style-to-use-legacyhttps://bugs.python.org/issue7117
However, all gcc architecture use a representation compliant with
IEE754. So, we can enable it unconditionnaly.
Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
[Thomas: adjust condition to avoid usage of qstrip, suggested by Baruch.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
During the execution of its configure script, Python tries to find an
available "hg" and "svn" installation, and if available, will try to use
them to get information from the version control system. To do this, it
tries to communicate over the network, potentially over ports that are
blocked, causing the build to halt. This was reported by a user as part
of bug #7802.
To solve this, we simply make the Python script use /bin/false as the
"hg" and "svn" programs.
Fixes bug #7802 for the python package.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Register package-specific target-finalize hooks with the
newly-introduced <PKG>_TARGET_FINALIZE_HOOKS.
This incidentally fixes luarocks, which was registering target-finalize
hooks even when it was not enabled.
To be noted, the skeleton package is not converted, because it is not
optional, we always have it; so its hooks would always be registered
anyway. Besides, the followup patches would render this conversion moot
anyway, since those hooks would be spread across the various skeleton
packages.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Rebased 011-remove-python-symlink.patch
[Peter: correct .hash file comment as pointed out by Baruch]
Signed-off-by: Bernd Kuhls <bernd.kuhls@t-online.de>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, each python package (be it the python interpreter package
itself or external python modules) is responsible for compiling its
.py into .pyc files. Unfortunately, this is not ideal as some packages
only install .py files without compiling them into .pyc files. In this
case, if the Buildroot configuration specifies to keep only the .pyc
files, the .py files are removed and lost.
To address this, this commit changes the logic by making the
compilation of .pyc files a global operation: the python interpreter
packages register a target finalize hook that is in charge of
compiling all installed .py files.
The *.pyc generation on a per package basis is disabled in the
python-package infrastructure by passing the "--no-compile" option to
setup.py.
The *.pyc generation for the Python interpreter internal modules is
disabled through --disable-pyc-build configure option.
A small helper script is used to perform the compilation, the purpose
of this script is to abort the compilation process if one of the .py
file cannot be compiled. It has been provided by Samuel Martin and
integrated into this commit.
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
[Thomas:
- rework for python 3.5
- integrate Samuel proposal that allows to detect compilation
failures.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
As suggested by Samuel Martin, this commit adds the option
--no-run-if-empty xargs option to the "find ... | xargs ..." logic used
in the python and python3 target-finalize hooks to remove py/pyc/pyo
files. This ensures that the command doesn't fail if there are no files
matching the pattern.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Even though we disable the build of .pyo files in the interpreter,
nothing prevents other packages to install them. Since we only want to
keep either .py or .pyc or both, let's add a target finalize hooks
that removes all .pyo files.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
[Thomas: add --no-run-if-empty option to xargs, as suggested by Samuel.]
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Since the bump of python3 to 3.5.x, the target finalize hooks
registered by the python3 have been changed a little bit. For the sake
of consistency, this commit aligns the target finalize hooks
registered by the python package so that they look the same as the
ones used by python3.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Patches are simply refreshed, except
004-sysconfigdata-install-location.patch where a minor conflict
resolution was needed.
[Peter: fixup .hash as pointed out by Arnout]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Now that we have a configure option in Python to enable/disable the
ossaudiodev module, this commit adds a configuration option to the
target Python to explicitly enable/disable this module.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This module is not needed to build the target Python, and can cause
some build issues on certain systems (when <linux/soundcard.h> does
not contain the OSS related definitions).
Signed-off-by: Zoltan Gyarmati <mr.zoltan.gyarmati@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
In addition to doing the bump, this commit also:
- Refreshes all the patches
- Removes python-003-properly-detect-if-python-build.patch, which has
been applied upstream.
- Passes the --without-ensurepip option, like is done in Python 3, to
avoid having Python use PIP to automatically download stuff when it
is being built.
- PYTHON_LIBTOOL_PATH = NO is added to prevent Buildroot from trying
to patch a version of libtool for which we don't have matching
patches, which isn't a problem since we're anyway not using the
part of the Python sources that uses libtool (it's the built-in
copy of libffi, and we use the external libffi).
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
The Buildroot coding style defines one space around make assignments and
does not align the assignment symbols.
This patch does a bulk fix of offending packages. The package
infrastructures (or more in general assignments to calculated variable
names, like $(2)_FOO) are not touched.
Alignment of line continuation characters (\) is kept as-is.
The sed command used to do this replacement is:
find * -name "*.mk" | xargs sed -i \
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*$#\1 \2#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\]\+\)$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\ \t]\+\s*\\\)\s*$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\(\s*\\\)#\1 \2\3#'
Brief explanation of this command:
^\([A-Z0-9a-z_]\+\) a regular variable at the beginning of the line
\([?:+]\?=\) any assignment character =, :=, ?=, +=
\([^\\]\+\) any string not containing a line continuation
\([^\\ \t]\+\s*\\\) string, optional whitespace, followed by a
line continuation character
\(\s*\\\) optional whitespace, followed by a line
continuation character
Hence, the first subexpression handles empty assignments, the second
handles regular assignments, the third handles regular assignments with
line continuation, and the fourth empty assignments with line
continuation.
This expression was tested on following test text: (initial tab not
included)
FOO = spaces before
FOO = spaces before and after
FOO = tab before
FOO = tab and spaces before
FOO = tab after
FOO = tab and spaces after
FOO = spaces and tab after
FOO = \
FOO = bar \
FOO = bar space \
FOO = \
GENIMAGE_DEPENDENCIES = host-pkgconf libconfuse
FOO += spaces before
FOO ?= spaces before and after
FOO :=
FOO =
FOO =
FOO =
FOO =
$(MAKE1) CROSS_COMPILE=$(TARGET_CROSS) -C
AT91BOOTSTRAP3_DEFCONFIG = \
AXEL_DISABLE_I18N=--i18n=0
After this bulk change, following manual fixups were done:
- fix line continuation alignment in cegui06 and spice (the sed
expression leaves the number of whitespace between the value and line
continuation character intact, but the whitespace before that could have
changed, causing misalignment.
- qt5base was reverted, as this package uses extensive alignment which
actually makes the code more readable.
Finally, the end result was manually reviewed.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Cc: Yann E. Morin <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Since the removal of py/pyc files is Python-specific, this commit
moves the logic removing those files to python.mk and python3.mk
respectively.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
To be consistent with the recent change of FOO_MAKE_OPT into FOO_MAKE_OPTS,
make the same change for FOO_CONF_OPT.
Sed command used:
find * -type f | xargs sed -i 's#_CONF_OPT\>#&S#g'
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This patch adds support for time.tzset() in Python 2.7 and Python 3.
The Python configure script sets ac_cv_working_tzset to 'no' when
cross compiling, although it should be available and work with all
toolchains supported by Buildroot (?).
The patch below pre-sets the variable so that time.tzset() works.
[Thomas: adjust commit log.]
Signed-off-by: Henry Margies <henry.margies@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The different patches are just refreshed to apply without fuzz, there
are no changes/conflicts.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
IDLE is the Python IDE built with the tkinter GUI toolkit, for which
the main script has always been removed from the target (so it was
never usable in Buildroot). However, we were still installing about
800 to 900 KB of .pyc files used only by the idle editor. This commit
adds a Python patch that completely gets rid of the
compilation/installation of the idle editor. As a consequence, it is
no longer needed to manually remove the 'idle' program from the
target in python.mk.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Status of patches:
- python-013-fix-readline-6.3.patch is removed, as it was a backport
from an upstream fix.
- all other patches are updated, with no special changes other than
solving minor conflicts.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This change adds a patch to python disabling the installation of the
python and python-config symlinks.
This allows Buildroot to control these symlinks' installation:
* the python symlink should be unconditionally installed in the target
tree, and the python-config symlink in the staging tree, since it is
only built and installed in the target tree if the user selected it;
* the python and python-config symlinks should only be installed in
the host tree when python(2) is the selection of the user for the
target.
Otherwise, when python3 is selected for the target, the host-python
may be required to built some packages. In such cases, the python
symlink should points to python3 (so should the python-config
symlink) to reflect the staging/target tree.
[Thomas: fix comments according to Yann's suggestions, and replaced
python(2) by python2, as suggested by Yann.]
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
During installation, host-python and host-python3 run the freshly built
python executable. This is done with a proper LD_LIBRARY_PATH to make
sure it picks up the libpython in the build directory. However, the
python binary has an RPATH pointing to the $(HOST_DIR)/usr/lib.
Therefore, if libpython exists there, it will be used instead.
If the install step is run in parallel, it is possible that libpython
is already partially copied to $(HOST_DIR)/usr/lib when python is run.
This gives an error like:
python: error while loading shared libraries: $(HOST_DIR)/usr/lib/libpython3.4m.so.1.0: file too short
The fix is simple: use RUNPATH instead of RPATH, which allows
LD_LIBRARY_PATH to override RUNPATH. That way, the libpython in the
build directory is always used. RUNPATH is enabled by passing
--enable-new-dtags to the linker.
Fixes e.g.
http://autobuild.buildroot.net/results/2a6/2a62de3247ba5ad273f03d01e690a3eeb11aa7b4
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
A confusion was made when applying commit
c3d539b53f ('host python/python3:
conditionally disable unicodedata') and then commit
73293e88c8 ('python, python3: enable
unicodedata for host-python, needed by setuptools').
The first commit added the enabling of unicodedata in the host Python
when the target Python is configured with unicodedata, which is
necessary because building the target unicodedata Python module
requires having unicodedata support in the host Python.
The second commit enabled the unicodedata unconditionally in the host
Python, because it is needed by host-setuptools.
However, when the second commit was applied, the logic that consisted
in disabling unicodedate in the host Python when not needed was kept,
which resulted in breaking the build of host-setuptools due to the
lack of unicodedata support in the host python.
This commit fixes that by only leaving the unconditional
--enable-unicodedata in the host Python configuration options, and
therefore removing the problematic conditional --disable-unicodedata.
Fixes:
http://autobuild.buildroot.org/results/b62/b622fc14776f5a6c26ff0d1fb882f4ae6cccc5d8/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
As we are going to bump setuptools to a much newer version, the host
python needs to be built with support for unicodedata.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The host python always had --disable-unicodedata, regardless of the
corresponding configuration option BR2_PACKAGE_PYTHON_UNICODEDATA.
Since the host python is used to byte-compile python modules, this meant
that such modules could not contain unicode strings. For example, following
statement in a python module:
print u"\N{SOLIDUS}"
would cause the byte-compilation to fail with message:
SyntaxError: ("(unicode error) \\N escapes not supported (can't load
unicodedata module)",
Instead, conditionally disable unicodedata based on
BR2_PACKAGE_PYTHON_UNICODEDATA, also for the host python.
This fixes bug #6542 (https://bugs.busybox.net/show_bug.cgi?id=6542)
Reported-by: Gernot Vormayr <gvormayr@gmail.com>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Even though jumping from 2.7.3 to 2.7.6 looks like a minor version
bump, it is in fact a fairly significant one, because a good number of
changes to help cross-compilation have been merged into Python
upstream. Therefore, most of our patches are affected by this change.
In detail, this commit:
* Renames all the patches to follow the naming convention of patches
in Buildroot: the patch file names should not have any version
number.
* The patches numbered above 100, that add configuration options to
disable certain modules of the Python standard library, are only
renamed and slightly adapted, they didn't change that much.
* The patches numbered below 100 are almost entirely rewritten: many
of the cross-compilation problems that used to exist in Python
2.7.3 no longer exist, and the number of remaining problems is
smaller, and can be fixed with smaller patches.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
As a preparation to make the Python infrastructure support both Python
and Python 3, as well as the bump of Python 2 and 3, we need the
Python package to expose the Python module path in a variable called
PYTHON_PATH. It will be used by the following commits.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This patch is based on the original new pkg patch submitted last Jan
and is part of the "Patchwork oldest patches cleanup #5".
[Peter: fix CONF_OPT indentation]
Signed-off-by: Matt Weber <mlweber1@rockwellcollins.com>
Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Since 97c687000 (pkg-autotools.mk: default host AUTORECONF{,_OPT} to the
target values) we automatically enable autoreconf for host builds if it
is enabled for the target, so these can go.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
[Peter: leave change xz tarball format to not end up with circular deps]
Signed-off-by: Jerzy Grzegorek <jerzy.grzegorek@trzebnica.net>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Also move smtpd.py removal to the global remove useless files define.
Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
This finally removes the BR2_HAVE_DEVFILES option, that was used to
install/keep development files on target. With the recent migration of
the internal backend to the package infrastructure, we had anyway lost
the ability to build gcc for the target, and install the uClibc
development files on the target.
[Peter: also remove support/scripts/copy.sh]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
We already remove python2.7-config and the symbolic link
python-config, but we forgot to remove the python2-config symbolic
link.
Note that we can't use the <pkg>_CONFIG_SCRIPTS mechanism here because
python2.7-config is written in... Python, and doesn't follow the usual
syntax of <pkg>-config scripts. It takes the paths directly from
distutils.sysconfig.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
It's mostly sample code, normally not used, and has a bad shebang line.
Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Commit 3c90f75496 made Python use a
special ./configure command in order to avoid --enable-shared
--disable-static being passed, because it was causing issues when
building certain modules for a 64 bits system.
However, not having a shared libpython2.7 library for the host
prevents the libxml2 Python binding to get built.
So instead, we use the default configure command, but we add
--enable-static which is needed for Python to build correctly.
Note that we tested the build of Python on a 64 bits host as well as
the build of Python for a 64 bits target, and both went fine, with all
modules built properly.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
