Buildroot generates Linux systems, so there is no need to have
MS Windows executables in there.
This reduces the target filesystem size by about 600kB:
$ du -hcs usr/lib/python2.7/distutils/command/wininst-*.exe
60K usr/lib/python2.7/distutils/command/wininst-6.0.exe
64K usr/lib/python2.7/distutils/command/wininst-7.1.exe
60K usr/lib/python2.7/distutils/command/wininst-8.0.exe
192K usr/lib/python2.7/distutils/command/wininst-9.0.exe
220K usr/lib/python2.7/distutils/command/wininst-9.0-amd64.exe
596K total
Signed-off-by: Titouan Christophe <titouan.christophe@railnova.eu>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a new option that prints the (runtime) path of compiled .py files
when VERBOSE=1 is set.
Signed-off-by: Robin Jarry <robin.jarry@6wind.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
When generating a .pyc file, the original .py source file path is
encoded in it. It is used for various purposes: traceback generation,
.pyc file comparison with its .py source, and code inspection.
By default, the source path used when invoking compileall is encoded in
the .pyc file. Since we use paths relative to TARGET_DIR, we end up with
paths that are only valid when relative to '/' encoded in the installed
.pyc files on the target.
This breaks code inspection at runtime since the original source path
will be invalid unless the code is executed from '/'.
Unfortunately, compileall cannot be forced to use the proper path. It
was not written with cross-compilation usage in mind.
Rework the script to call py_compile.compile() directly with pertinent
options:
- The script now has a new --strip-root argument. This argument is
optional but will always be specified when compiling py files in
buildroot.
- All other (non-optional) arguments are folders in which all
"importable" .py files will be compiled to .pyc.
- Using --strip-root=$(TARGET_DIR), the future runtime path of each .py
file is computed and encoded into the compiled .pyc.
No need to change directory before running the script anymore.
The trickery used to handle error reporting was only applicable with
compileall. Since we implement our own "compileall", error reporting
becomes trivial.
Previously, we had a --force option to tell compileall.compiledir() to
forcibly recompile files if they had changed. Now, we would have to
handle it ourselves. It turns out to not be easy and would need us to
delve into the format of bytecompiled files to extract metadata and
compare it with the expected values, that being even dependent on the
python version being used (fortunately, only two for us: python 2.7 and
the latext 3.x).
Still, this is deemed too complex, and byte-compiling is pretty fast, so
much so that it should be eclipsed by the build duration anyway.
So we just drop support for --force, and instead we always byte-compile.
Signed-off-by: Julien Floret <julien.floret@6wind.com>
Signed-off-by: Robin Jarry <robin.jarry@6wind.com>
[yann.morin.1998@free.fr:
- always byte-compile
- drop --force
- expand commit log to state so and explain why
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
With per-package directory support, Python external modules are
causing a problem: the _sysconfigdata.py module installed by the
Python interpreter contains a number of paths that are relative to the
current package per-package directory, i.e python or python3. For
example:
'BLDSHARED': '/home/thomas/projets/buildroot/output/per-package/python/host/bin/arm-linux-gcc -shared',
'CC': '/home/thomas/projets/buildroot/output/per-package/python/host/bin/arm-linux-gcc',
'CXX': '/home/thomas/projets/buildroot/output/per-package/python/host/bin/arm-linux-g++',
etc.
These paths are problematic, because it means that the wrong compiler
gets used when building external Python modules: instead of using the
compiler from the external Python module per-package host directory,
it uses the one from the 'python' or 'python3' per-package host
directory. Due to this, any native dependency needed by the external
Python module is not found, even though it is properly present in the
current package per-package directory.
Of course, the problem occurs with both target Python modules and host
Python modules.
To fix this, we simply rewrite those paths in _sysconfigdata.py before
building a Python package.
Interestingly, until now, the _sysconfidata.py that was used during
the build was the one from $(TARGET_DIR), which is a bit unusual: it
is more common to use files from $(STAGING_DIR) during the build
process. So this commit changes the PYTHON_PATH and PYTHON3_PATH
variables so that they point to $(STAGING_DIR), which makes the
_sysconfigdata.py fixup in $(STAGING_DIR) effective.
Fixes:
http://autobuild.buildroot.net/results/a24b0555fd4261b50dc3986635c30717d9cbe764/ (python-psycopg2)
http://autobuild.buildroot.net/results/080fa893e1b0e7a8c8a31ac1c98eb8871b97264d/ (python-alsaaudio)
http://autobuild.buildroot.net/results/79bc070f98d6d9d8ef78df12b248cdc7d0e405c3/ (python-lxml)
and many more Python packages that use native code with a native library
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Fixes:
http://autobuild.buildroot.net/results/3b6/3b6280b0b7a9634b747db2865b21c6266007c725/
The PYTHON_KEEP_PY_FILES global variable conflicts with the per-package
<pkg>_KEEP_PY_FILES variable for the python package, causing make to
complain:
package/zlib/zlib.mk:7: *** Recursive variable 'PYTHON_KEEP_PY_FILES' references itself (eventually). Stop.
As a workaround, rename the global variable to KEEP_PYTHON_PY_FILES so it
cannot conflict with the per-package variable.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
When BR2_PACKAGE_PYTHON{,3}_PYC_ONLY=y, we force remove all .py files
from the system, as they have all been byte-compiled into their .pyc
variants.
However, it turns out that some packages (e.g: OpenCV) do some funky
things with a few .py files: they pass them through Python's
execfile() facility, which only works with .py files and not .pyc
files. It is used by OpenCV for example to read two small
configuration files.
In order to support such use cases, this commit introduces a very
simple mechanism by which packages can exclude some path patterns from
the .py removal: a per-package <pkg>_KEEP_PY_FILES variable that is
collected into a global PYTHON_KEEP_PY_FILES variable, then used by
the python/python3 target-finalize hooks.
This variable is intentionally not documented, this is really a hack
that we ideally would like to see go away, and we'd rather not see its
usage spread too much.
This is necessary to be able to fix bug #12171.
[Peter: check if PYTHON_KEEP_PY_FILES contains non-white space]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Some python scripts may be ran in the custom scripts a user can define
in the config. Allow the user to enable host-python explicitly.
If any of those require ssl, they will fail with no possible fix.
Add an option to enable openssl as well. This is made optional because
openssl significantly increases the build time.
Signed-off-by: Nicolas Carrier <nicolas.carrier@orolia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This release fixes CVE-2019-9740, CVE-2019-9948, CVE-2019-15903.
Adjust 0002-Fix-get_python_inc-for-cross-compilation.patch for 2.7.17.
Remove the following patches (now on upstream):
* 0035-bpo-35907-CVE-2019-9948-urllib-rejects-local_file-sc.patch
* 0036-bpo-36216-Add-check-for-characters-in-netloc-that-no.patch
* 0037-3.7-bpo-36216-Only-print-test-messages-when-verbose-.patch
* 0038-bpo-36742-Fixes-handling-of-pre-normalization-charac.patch
* 0039-bpo-36742-Corrects-fix-to-handle-decomposition-in-us.patch
* 0040-2.7-bpo-36742-Fix-urlparse.urlsplit-error-message-fo.patch
* 0041-bpo-30458-Disallow-control-chars-in-http-URLs-GH-127.patch
Full release details at:
https://github.com/python/cpython/blob/v2.7.17/Misc/NEWS.d/2.7.17rc1.rst
run-tests results:
10:30:20 TestPython2 Starting
10:30:21 TestPython2 Building
10:37:37 TestPython2 Building done
10:37:47 TestPython2 Cleaning up
.
----------------------------------------------------------------------
Ran 1 test in 448.616s
OK
Signed-off-by: Asaf Kahlon <asafka7@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
An issue was discovered in urllib2 in Python 2.x through 2.7.16 and urllib
in Python 3.x through 3.7.3. CRLF injection is possible if the attacker
controls a url parameter, as demonstrated by the first argument to
urllib.request.urlopen with \r\n (specifically in the query string after a ?
character) followed by an HTTP header or a Redis command.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Fixes the following security issues:
- CVE-2013-1752: Change use of readline() in :class:`imaplib.IMAP4_SSL` to limit line length
- CVE-2018-14647: The C accelerated _elementtree module now initializes hash
randomization salt from _Py_HashSecret instead of libexpat's default
CSPRNG.
For more details, see the NEWS file:
https://github.com/python/cpython/blob/v2.7.16/Misc/NEWS.d/2.7.16rc1.rst
Refresh patches, drop now upstream
package/python/0035-bpo-35746-Fix-segfault-in-ssl-s-cert-parser-GH-11569.patch
and adjust hash of LICENSE file for a change of copyright years.
run-tests results:
16:05:41 TestPython2 Starting
16:05:42 TestPython2 Building
16:11:26 TestPython2 Building done
16:11:32 TestPython2 Cleaning up
.
----------------------------------------------------------------------
Ran 1 test in 351.905s
OK
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This commit fixes the warnings reported by check-package on the help
text of all package Config.in files, related to the formatting of the
help text: should start with a tab, then 2 spaces, then at most 62
characters.
The vast majority of warnings fixed were caused by too long lines. A
few warnings were related to spaces being used instead of a tab to
indent the help text.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
We currently have
$(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/site-packages/
inside the PYTHON_PATH variable, which gets used to define PYTHONPATH,
passed to the host Python interpreter when building/installing target
packages.
However, this is terribly wrong, as it causes the host interpreter to
potentially import target Python packages. This is wrong for several
reasons:
- Some Python packages might need some Python modules to be installed
on the host (described in setup_requires in setup.py), but their
installation currently works because by luck the corresponding
Python module is installed for the target. Some of those cases were
happening for real, and fixed by previous patches.
- Some Python packages include some native code, therefore built for
a specific CPU architecture. When you point the host Python
interpreter to native libraries built for the target, you get nice
build failures, such as the one affecting the python-cffi related
packages.
Making this change allows to fix the python-cffi related build
failures:
http://autobuild.buildroot.net/results/a9af84f2d845ee25e2b7d8b92aef485112b46060/
(python-cryptography)
http://autobuild.buildroot.net/results/b017c4f6b4d45c0afbf06a80dbd3f2ebe5d49d20/
(python-pynacl)
http://autobuild.buildroot.net/results/25144ea191ad46d851b31d3a2f0ef939f215494b/
(python-smbus-cffi)
This change has been verified with the following defconfig that
enables a lot of Python packages:
BR2_arm=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arm-full-2017.05-834-gb595627.tar.bz2"
BR2_TOOLCHAIN_EXTERNAL_GCC_4_9=y
BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_10=y
BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
BR2_TOOLCHAIN_EXTERNAL_CXX=y
BR2_INIT_NONE=y
BR2_SYSTEM_BIN_SH_NONE=y
BR2_PACKAGE_PYTHON=y
BR2_PACKAGE_PYTHON_ALSAAUDIO=y
BR2_PACKAGE_PYTHON_ARROW=y
BR2_PACKAGE_PYTHON_ATTRS=y
BR2_PACKAGE_PYTHON_AUTOBAHN=y
BR2_PACKAGE_PYTHON_BITSTRING=y
BR2_PACKAGE_PYTHON_BOTTLE=y
BR2_PACKAGE_PYTHON_CAN=y
BR2_PACKAGE_PYTHON_CBOR=y
BR2_PACKAGE_PYTHON_CHARDET=y
BR2_PACKAGE_PYTHON_CHEETAH=y
BR2_PACKAGE_PYTHON_CHERRYPY=y
BR2_PACKAGE_PYTHON_CONFIGOBJ=y
BR2_PACKAGE_PYTHON_CONFIGSHELL_FB=y
BR2_PACKAGE_PYTHON_CRC16=y
BR2_PACKAGE_PYTHON_CRCMOD=y
BR2_PACKAGE_PYTHON_CSSSELECT=y
BR2_PACKAGE_PYTHON_CSSUTILS=y
BR2_PACKAGE_PYTHON_DAEMON=y
BR2_PACKAGE_PYTHON_DIALOG=y
BR2_PACKAGE_PYTHON_DICTTOXML=y
BR2_PACKAGE_PYTHON_DJANGO=y
BR2_PACKAGE_PYTHON_DOCOPT=y
BR2_PACKAGE_PYTHON_DPKT=y
BR2_PACKAGE_PYTHON_ECDSA=y
BR2_PACKAGE_PYTHON_ENUM=y
BR2_PACKAGE_PYTHON_FLASK_BABEL=y
BR2_PACKAGE_PYTHON_FLASK_JSONRPC=y
BR2_PACKAGE_PYTHON_FLASK_LOGIN=y
BR2_PACKAGE_PYTHON_FLUP=y
BR2_PACKAGE_PYTHON_GOBJECT=y
BR2_PACKAGE_PYTHON_GUNICORN=y
BR2_PACKAGE_PYTHON_HTML5LIB=y
BR2_PACKAGE_PYTHON_HTTPLIB2=y
BR2_PACKAGE_PYTHON_HUMANIZE=y
BR2_PACKAGE_PYTHON_ID3=y
BR2_PACKAGE_PYTHON_INIPARSE=y
BR2_PACKAGE_PYTHON_IOWAIT=y
BR2_PACKAGE_PYTHON_IPADDR=y
BR2_PACKAGE_PYTHON_IPY=y
BR2_PACKAGE_PYTHON_IPYTHON=y
BR2_PACKAGE_PYTHON_JSON_SCHEMA_VALIDATOR=y
BR2_PACKAGE_PYTHON_KEYRING=y
BR2_PACKAGE_PYTHON_LIBCONFIG=y
BR2_PACKAGE_PYTHON_LMDB=y
BR2_PACKAGE_PYTHON_LXML=y
BR2_PACKAGE_PYTHON_MAD=y
BR2_PACKAGE_PYTHON_MARKDOWN=y
BR2_PACKAGE_PYTHON_MELD3=y
BR2_PACKAGE_PYTHON_MISTUNE=y
BR2_PACKAGE_PYTHON_MSGPACK=y
BR2_PACKAGE_PYTHON_MUTAGEN=y
BR2_PACKAGE_PYTHON_MWSCRAPE=y
BR2_PACKAGE_PYTHON_NETADDR=y
BR2_PACKAGE_PYTHON_NETIFACES=y
BR2_PACKAGE_PYTHON_NFC=y
BR2_PACKAGE_PYTHON_NUMPY=y
BR2_PACKAGE_PYTHON_PAHO_MQTT=y
BR2_PACKAGE_PYTHON_PAM=y
BR2_PACKAGE_PYTHON_PARAMIKO=y
BR2_PACKAGE_PYTHON_PILLOW=y
BR2_PACKAGE_PYTHON_POSIX_IPC=y
BR2_PACKAGE_PYTHON_PSUTIL=y
BR2_PACKAGE_PYTHON_PUDB=y
BR2_PACKAGE_PYTHON_PYCLI=y
BR2_PACKAGE_PYTHON_PYCPARSER=y
BR2_PACKAGE_PYTHON_PYELFTOOLS=y
BR2_PACKAGE_PYTHON_PYFTPDLIB=y
BR2_PACKAGE_PYTHON_PYGAME=y
BR2_PACKAGE_PYTHON_PYGAME_IMAGE=y
BR2_PACKAGE_PYTHON_PYGAME_EXAMPLES=y
BR2_PACKAGE_PYTHON_PYGAME_FONT=y
BR2_PACKAGE_PYTHON_PYGAME_MIXER=y
BR2_PACKAGE_PYTHON_PYINOTIFY=y
BR2_PACKAGE_PYTHON_PYLIBFTDI=y
BR2_PACKAGE_PYTHON_PYMYSQL=y
BR2_PACKAGE_PYTHON_PYPARTED=y
BR2_PACKAGE_PYTHON_PYPCAP=y
BR2_PACKAGE_PYTHON_PYQRCODE=y
BR2_PACKAGE_PYTHON_PYRATEMP=y
BR2_PACKAGE_PYTHON_PYRO=y
BR2_PACKAGE_PYTHON_PYROUTE2=y
BR2_PACKAGE_PYTHON_PYSENDFILE=y
BR2_PACKAGE_PYTHON_PYSMB=y
BR2_PACKAGE_PYTHON_PYSNMP_APPS=y
BR2_PACKAGE_PYTHON_PYSNMP_MIBS=y
BR2_PACKAGE_PYTHON_PYSOCKS=y
BR2_PACKAGE_PYTHON_PYTABLEWRITER=y
BR2_PACKAGE_PYTHON_PYTRIE=y
BR2_PACKAGE_PYTHON_PYUSB=y
BR2_PACKAGE_PYTHON_PYXB=y
BR2_PACKAGE_PYTHON_PYZMQ=y
BR2_PACKAGE_PYTHON_REQUESTS_TOOLBELT=y
BR2_PACKAGE_PYTHON_RPI_GPIO=y
BR2_PACKAGE_PYTHON_RTSLIB_FB=y
BR2_PACKAGE_PYTHON_SDNOTIFY=y
BR2_PACKAGE_PYTHON_SERIAL=y
BR2_PACKAGE_PYTHON_SETPROCTITLE=y
BR2_PACKAGE_PYTHON_SH=y
BR2_PACKAGE_PYTHON_SHUTILWHICH=y
BR2_PACKAGE_PYTHON_SIMPLEJSON=y
BR2_PACKAGE_PYTHON_SMBUS_CFFI=y
BR2_PACKAGE_PYTHON_SOCKETIO=y
BR2_PACKAGE_PYTHON_SORTEDCONTAINERS=y
BR2_PACKAGE_PYTHON_SPIDEV=y
BR2_PACKAGE_PYTHON_THRIFT=y
BR2_PACKAGE_PYTHON_TOMAKO=y
BR2_PACKAGE_PYTHON_TREQ=y
BR2_PACKAGE_PYTHON_U_MSGPACK=y
BR2_PACKAGE_PYTHON_UBJSON=y
BR2_PACKAGE_PYTHON_UJSON=y
BR2_PACKAGE_PYTHON_URLLIB3=y
BR2_PACKAGE_PYTHON_VERSIONTOOLS=y
BR2_PACKAGE_PYTHON_WATCHDOG=y
BR2_PACKAGE_PYTHON_WEB2PY=y
BR2_PACKAGE_PYTHON_WEBPY=y
BR2_PACKAGE_PYTHON_WHOOSH=y
BR2_PACKAGE_PYTHON_WS4PY=y
BR2_PACKAGE_PYTHON_WSACCEL=y
BR2_PACKAGE_PYTHON_XLUTILS=y
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Since things are no longer installed in $(HOST_DIR)/usr, the callers
should also not refer to it.
This is a mechanical change with
git grep -l '$(HOST_DIR)/usr/bin' | xargs sed -i 's%$(HOST_DIR)/usr/bin%$(HOST_DIR)/bin%g'
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit switches to use the new gettext logic, which involves
using TARGET_NLS_DEPENDENCIES instead of hand-encoded dependencies
on gettext/host-gettext.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The check-package script when ran gives warnings on ordering issues
on all of these Config files. This patch cleans up all warnings
related to the ordering in the Config files for packages starting with
the letter p in the package directory.
The appropriate ordering is: type, default, depends on, select, help
See http://nightly.buildroot.org/#_config_files for more information.
Signed-off-by: Adam Duskett <Adamduskett@outlook.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The check-package script when ran gave warnings on only using
one space before backslashes on all of these makefiles.
This patch cleans up all warnings related to the one space before
backslashes rule in the make files in the package directory.
Signed-off-by: Adam Duskett <aduskett@codeblue.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
.pyc files include path to source .py file. This patch changes the way
`pycompile.py' is launched in order to only keep the part relative to
$TARGET_DIR.
This work was sponsored by `BA Robotic Systems'.
Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
As noticed by André Hentschel <nerv@dawncrow.de>, commit
6520762932 ("python: move to Git formatted
patches") mistakenly removed 018-fix-add-gcc-paths-logic.patch.
This causes bug #7971 to re-appear. To fix this, we re-introduce the
missing patch.
Reported-by: André Hentschel <nerv@dawncrow.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Whitespaces were searched using the following regex:
[ ]{1,}\t
and then manually removed in most of the cases. For
xserver_xorg-server.mk, tabs before backslashes were removed.
Signed-off-by: Bernd Kuhls <bernd.kuhls@t-online.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
.pyc files contain the modification time of the corresponding .py
source. In order to make the build reproducible, we fix the modification
time of all .py before compiling .pyc files.
In addition, since pycompile relies on the modification time to know if
a file needs to be recompiled, it is safer to force recompilation of all
source files.
This work was sponsored by `BA Robotic Systems'.
Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
[Thomas: do not register PYTHON_FIX_TIME as a
PYTHON_TARGET_FINALIZE_HOOKS, instead call it inside
PYTHON_CREATE_PYC_FILES before doing the byte compilation.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-elecrons.com>
The BR2_PACKAGE_PYTHON_{READLINE,HASHLIB} options were so far only
bringing in the necessary dependencies, relying on the Python build
system to automatically detect them.
However, this means that even if one of those option was disabled, if
their dependency was found, Python would build the corresponding module,
which is really not what the user would expect.
For example, if you have:
BR2_PACKAGE_READLINE=y
# BR2_PACKAGE_PYTHON_READLINE is not set
Then you would still get the readline Python module built and installed.
This commit fixes that by adding new --{enable,disable} options, and use
them in python.mk.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Now that the cpython project has a nice Github repository, with tags,
it's much nicer to handle the stack of Python patches with Git. The
python3 package patches had already been converted, but not the python
package patches. Therefore, this commit does the move.
There is no functional change, only reformatting of the patches.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently the host build of Python 2 defaults to narrow unicode (UCS2),
ignoring the BR2_PACKAGE_PYTHON_UCS4 configuration option which may be
set to wide (UCS4).
This results in host and target Python packages which are incompatible
in subtle ways.
For example, installing wheels into the target fails when they are made
with the host python, citing incompatibility (as can be seen by the
package tags which may be "cp27u-manylinux1" instead of
"cp27mu-manylinux1").
Compiling the host Python 2 with the same UCS configuration as the
target ensures that the packages are compatible (and the tags match).
This does not affect Python 3 as support for narrow unicode was
deprecated in version 3.3, see https://www.python.org/dev/peps/pep-0393/
Thanks to Tony Breeds <tony@bakeyournoodle.com> for reporting this.
Signed-off-by: Chris Smart <mail@csmart.io>
[Thomas: add comment in the code.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Python is not able to detect if compiler double representation is
compliant with IEE754:
checking whether C doubles are little-endian IEEE 754 binary64... no
checking whether C doubles are big-endian IEEE 754 binary64... no
checking whether C doubles are ARM mixed-endian IEEE 754 binary64... no
Accordingly 'legacy' mode isused. It is possible to check this at
runtime by check if 'sys.float_repr_style' contains 'short' or
'legacy'. Calculus correctness is not garanteed with 'legacy'.
Problem is better described here:
http://stackoverflow.com/questions/29920294/what-causes-pythons-float-repr-style-to-use-legacyhttps://bugs.python.org/issue7117
However, all gcc architecture use a representation compliant with
IEE754. So, we can enable it unconditionnaly.
Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
[Thomas: adjust condition to avoid usage of qstrip, suggested by Baruch.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
During the execution of its configure script, Python tries to find an
available "hg" and "svn" installation, and if available, will try to use
them to get information from the version control system. To do this, it
tries to communicate over the network, potentially over ports that are
blocked, causing the build to halt. This was reported by a user as part
of bug #7802.
To solve this, we simply make the Python script use /bin/false as the
"hg" and "svn" programs.
Fixes bug #7802 for the python package.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Register package-specific target-finalize hooks with the
newly-introduced <PKG>_TARGET_FINALIZE_HOOKS.
This incidentally fixes luarocks, which was registering target-finalize
hooks even when it was not enabled.
To be noted, the skeleton package is not converted, because it is not
optional, we always have it; so its hooks would always be registered
anyway. Besides, the followup patches would render this conversion moot
anyway, since those hooks would be spread across the various skeleton
packages.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Rebased 011-remove-python-symlink.patch
[Peter: correct .hash file comment as pointed out by Baruch]
Signed-off-by: Bernd Kuhls <bernd.kuhls@t-online.de>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, each python package (be it the python interpreter package
itself or external python modules) is responsible for compiling its
.py into .pyc files. Unfortunately, this is not ideal as some packages
only install .py files without compiling them into .pyc files. In this
case, if the Buildroot configuration specifies to keep only the .pyc
files, the .py files are removed and lost.
To address this, this commit changes the logic by making the
compilation of .pyc files a global operation: the python interpreter
packages register a target finalize hook that is in charge of
compiling all installed .py files.
The *.pyc generation on a per package basis is disabled in the
python-package infrastructure by passing the "--no-compile" option to
setup.py.
The *.pyc generation for the Python interpreter internal modules is
disabled through --disable-pyc-build configure option.
A small helper script is used to perform the compilation, the purpose
of this script is to abort the compilation process if one of the .py
file cannot be compiled. It has been provided by Samuel Martin and
integrated into this commit.
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
[Thomas:
- rework for python 3.5
- integrate Samuel proposal that allows to detect compilation
failures.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
As suggested by Samuel Martin, this commit adds the option
--no-run-if-empty xargs option to the "find ... | xargs ..." logic used
in the python and python3 target-finalize hooks to remove py/pyc/pyo
files. This ensures that the command doesn't fail if there are no files
matching the pattern.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Even though we disable the build of .pyo files in the interpreter,
nothing prevents other packages to install them. Since we only want to
keep either .py or .pyc or both, let's add a target finalize hooks
that removes all .pyo files.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
[Thomas: add --no-run-if-empty option to xargs, as suggested by Samuel.]
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Since the bump of python3 to 3.5.x, the target finalize hooks
registered by the python3 have been changed a little bit. For the sake
of consistency, this commit aligns the target finalize hooks
registered by the python package so that they look the same as the
ones used by python3.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Patches are simply refreshed, except
004-sysconfigdata-install-location.patch where a minor conflict
resolution was needed.
[Peter: fixup .hash as pointed out by Arnout]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The Python setup.py has a function called add_gcc_paths(), which
executes gcc -E -v to get the list of header paths searched by
gcc. However, the logic of setup.py is only valid with the normal
english output of gcc: it doesn't work if a non-english locale is
set. This causes setup.py to not find certain headers (such as zlib.h)
and therefore disabling the build of such extensions.
Reported-by: Bruno Coudoin <bruno.coudoin@gcompris.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
