Metztli I. T. Blog

Posted by on Oct 16 2019 in Linux, Current Trends, Reiser4, Debian, php7

Build PHP 7.3.9̶ 10 ZTS and puppies...er, pthreads, on Metztli Reiser4: Как же мало нужно для счастья.

Prior to European invasion of Ixachitlan¹ there were xoloitzcuintzintin² –not ‘puppies’…

Although on special occasions and/or ceremonies the xoloitzcuintli was eaten by the olmecah, Toltecah –and later the Mexicah – it was during and after the Spanish monarchy’s sponsored invasion, when Spain’s genetic waste and their analogous symbiotic parasite rats infested Mexico-Tenochtitlan and devoured the xoloitzcuintzintin with such voracity the unique canine specimens were almost driven into extinction. Alarmed at the Spaniards’ rapacious craving for xoloitzcuintzintin meat, a number of xoloitzcuintli specimens either fled to or where surreptitiously taken to Mixtecapan –nowdays Oaxaca– defining a do-or-die inflection point to spare the xoloitzcuintli from total annihilation by the voracious alien Tzitzimimeh –embodied in the Spanish fanatical Christian terrorists. Conniving and co-opting military, logistic, and tactical support from jaundiced subjugated native ‘city-states’, Spanish scum gloated³ over their divide-and-conquer strategy to achieve ensuing destruction and pillage of the Mexicah’s highly sophisticated and cultured eponymous Altepetl (’city state’). Fact is, occupier Iberian State (IS) terrorist cliques even have appropriated ‘Mexican‘ – transliterated into their Spanish dialect by concatenating suffixes ‘-o‘ or ‘-a‘ for male or female, respectively – as their ‘ethnic‘ moniker; thus besmirching the legitimate ancient and surviving Mexicah / Nahua ethnic genotype with current Iberian unabated corruption, lawlessness, plundering, and low intensity environmental, economic, cultural, and ethnic genocide, all of which had been gradually building up since 1521.

⁴
Hmmm, pardon my memory focus lapse, I meant to begin with…

Realizing that the PHP developer of pthreads has, using the proverbial expression, ‘thrown his puppy…er, baby, out with the bathwater‘⁵, I had second thoughts about writing this post. I mean, to build the latest PHP 7.3.9̶ 10 the ‘Debian way‘ for Metztli Reiser4 / Debian Buster BPs was the only topic I was gonna annotate as my reference post. Notwithstanding, online I read of people literally ‘twerking‘ with phpize to build pthreads support for their PHP ZTS build and I thought: the ‘Debian way‘ really simplifies the procedure. What I mean is, after we hack the Debian packaging for PHP into producing a KrakJoe⁶ pthreads extension, all we have to do is install relevant DEB package and…that’s it! to enable the much sought after feature. I am not an expert on Toltecayotl –much less pthreads – so we will recycle a couple small test ‘pthreaded‘ programs, execute them, and that –hopefully– will be a measure of our success

NOTE: PHP 7.3.11 Fixes CVE-2019-11043, thus upgraded resources in this post. All references to PHP 7.3.10 apply equally to PHP 7.3.11 –including the patches I create here. (10-27-2019)
See https://bugs.php.net/bug.php?id=78599

Build PHP 7.3.10 the ‘Debian way’ on Metztli Reiser4 / Debian 10 Buster

We fetch PHP 7.3.10 and corresponding asc extension file to verify source integrity:

Shell

wget https://www.php.net/distributions/php-7.3.10.tar.xz https://www.php.net/distributions/php-7.3.10.tar.xz.asc
gpg --verify php-7.3.10.tar.xz.asc php-7.3.10.tar.xz

For our Debian Buster backports we will be using Debian Sid packaging for PHP 7.3.10-1; maintainer has caught up!

Shell

wget http://deb.debian.org/debian/pool/main/p/php7.3/php7.3_7.3.10-1.debian.tar.xz

Subsequently, create build directory and untar PHP 7.3.10 and Debian packaging for 7.3.10-1 into the former untarred source:

Shell

mkdir --verbose build && tar -C build -xPf php-7.3.10.tar.xz
tar -C build/php-7.3.10 -xPf php7.3_7.3.10-1.debian.tar.xz

For Debian 10 Buster we will be installing the same package names as in prior relevant blog post for Debian Stretch – thus satisfying our PHP 7.3.10 build requirements. Please review if you want to install the latest MariaDB available from MariaDB.org repositories, instead of Debian’s:

Shell

apt-get update
apt-get install install autoconf automake bison chrpath debhelper libmariadb-dev-compat dh-apache2 dpkg-dev flex freetds-dev libapparmor-dev libapr1-dev libargon2-0-dev libbz2-dev libc-client2007e-dev libcurl4-openssl-dev libdb-dev libedit-dev libenchant-dev libevent-dev libexpat1-dev libfreetype6-dev libgcrypt20-dev libgd-dev libglib2.0-dev libgmp3-dev libicu-dev libjpeg-dev libkrb5-dev libldap2-dev liblmdb-dev libmagic-dev libmhash-dev libnss-myhostname libonig-dev libpam0g-dev libpcre2-dev libpng-dev libpq-dev libpspell-dev libqdbm-dev librecode-dev libsasl2-dev libsnmp-dev libsodium-dev libsqlite3-dev libssl-dev libtidy-dev libtool libwebp-dev libwrap0-dev libxml2-dev libxmlrpc-epi-dev libxmltok1-dev libxslt1-dev libzip-dev locales-all netbase netcat-traditional re2c systemtap-sdt-dev tzdata unixodbc-dev zlib1g-dev dh-systemd libsystemd-dev firebird-dev mariadb-server

The list above is derived from debian/control file in Debian packaging for php7.3_7.3.10-1 on AMD64 and we only get a couple of ‘Notes‘:

Note, selecting ‘apache2-dev’ instead of ‘dh-apache2′
Note, selecting ‘libargon2-dev’ instead of ‘libargon2-0-dev’

We change directory to PHP 7.3.10 source, if we are not already there yet:

Shell

cd build/php-7.3.10/

create a link to debian/patches:

Shell

ln -s debian/patches

and apply the patches with quilt:

Shell

quilt push -a --fuzz=0

all patches should apply flawlessly –as seen in the snapshot taken at the end of the quilt procedure:

NOTE: If we are leveraging Debian packaging for php7.3_7.3.10-1.debian.tar.xz to build upstream PHP 7.3.11, we need to upgrade the version; otherwise, the generated packages will have lower incorrect version. Thus we need to correct the debian/changelog with:

Shell

dch -v 7.3.11-1

We may begin our PHP 7.3.10 build with the following directive:

Shell

dpkg-buildpackage -F -us -uc -jX -T binary-arch,binary-indep

From relevant man page, where X:= Number of jobs allowed to be run simultaneously, number of jobs matching the number of online processors if auto is specified (since dpkg 1.17.10), or unlimited number if jobs is not specified, equivalent to the make(1) option of the same name (since dpkg 1.14.7, long option since dpkg 1.18.8).

After thirty(30) to forty-five(45) minutes the Debian packaging for php7.3_7.3.10-1.1 build ends:

…and corresponding SHA256SUMs:

Build PHP 7.3.10 ZTS and pthreads the ‘Debian way’ on Metztli Reiser4

We go back a couple of directories – to our initial location where we downloaded relevant PHP resources – and create our build directory for PHP 7.3.10 ZTS and pthreads:

Shell

cd ../.. && mkdir --verbose build-zts-pthreads

As before, we untar PHP 7.3.10 source into the above directory we have just created:

Shell

tar -C build-zts-pthreads -xPf php-7.3.10.tar.xz

Listing PHP 7.3.10 source content:

Shell

ls build-zts-pthreads/php-7.3.10

we change into the PHP 7.3.10 source directory:

Shell

cd build-zts-pthreads/php-7.3.10/

As I am intending to make a patch, I will initiate with git:

Shell

git init

add everything:

Shell

git add .

and make Ynic cecni, i.e., First, commit:

Shell

git commit -a -m "Ynic cecni commit for PHP 7.3.10"

I list ext directory contents and change into it:

Shell

ls ext and cd ext

then clone krakjoe pthreads at that location:

Shell

git clone https://github.com/krakjoe/pthreads.git

By listing hidden git resources in newly cloned pthreads directory, I locate resource targets to remove:

Shell

ls pthreads/.travis.yml pthreads/.git

and remove those:

Shell

rm -rf pthreads/.travis.yml pthreads/.git

then I locate php_pthreads.h which I will edit in-place subsequently using sed:

Shell

ls pthreads/php_pthreads.h

as I am going to operate on a line range, excluding 30 but including 51, where I will replace all < less than and > greater than operator symbols by double quotes:

Shell

sed -i '30,51{s/[<,>]/"/g}' pthreads/php_pthreads.h

We can see the before and after the sed operation differences on referenced lines:

fact is our build will halt if those modifications are not done.

I will now be creating a metztli-ext-pthreads.patch that I will subsequently integrate into Debian packaging for php7.3_7.3.10-1 at debian/patches/metztli/metztli-ext-pthreads.patch.

First I go back from current directory ext/ to PHP 7.3.10 source root directory:

Shell

cd ..
git add .

and create an Ome, i.e., Second, commit for pthreads source addition:

Shell

git commit -a -m "Ome commit which added krakjoe phtreads into PHP 7.3.10 ext directory"

and create my patch:

Shell

git format-patch --stdout -1 >../metztli-ext-pthreads.patch

Modifying and simultaneously creating a ZTS and pthreads patch for Debian packaging for php7.3_7.3.10-1 itself.

Observing the image above, pthreads arguably was in the discipline of Toltecayotl, taught by the Toltecah masters –tlamemeh, i.e., carriers, of the xiuhcohuanahualli⁷ which enabled Tlalnepantla [’mesoamerica‘] civilization to flourish for many multiple millennia.

Picking up where I left off, I was inside source directory for PHP 7.3.10 with pthreads, and change one directory back:

Shell

cd ..
ls ..

I have Debian packaging php7.3_7.3.10-1.debian.tar.xz which when untarred will create a single debian directory were I will be modifying and/or adding resources so that it can hack out a PHP 7.3.10-1.1 ZTS with pthreads version of Debian packages.

Shell

tar -xPf ../php7.3_7.3.10-1.debian.tar.xz
cd debian

Again, I will be creating a patch simultaneously as I modify and/or add resources:

Shell

git init
git add .
git commit -a -m "Ynic cecni commit for Debian packaging for php7.3_7.3.10-1"

First file we will be modifying in-place is control –remember we are inside the debian directory. At line number 316, we will be adding following directives specifying creation of a pthreads module:

Code

Package: php7.3-pthreads
Architecture: any
Depends: php-common (>= 1:35),
         ucf,
         ${misc:Depends},
         ${php:Depends},
         ${php-pthreads:Depends},
         ${shlibs:Depends}
Pre-Depends: ${misc:Pre-Depends},
             ${php-pthreads:Pre-Depends}
Built-Using: ${php:Built-Using}
Replaces: ${php-pthreads:Replaces}
Breaks: ${php-pthreads:Breaks}
Conflicts: ${php-pthreads:Conflicts}
Provides: ${php-pthreads:Provides}
Recommends: ${php-pthreads:Recommends}
Description: KrakJoe pthreads module for PHP
 This package provides the KrakJoe pthreads module(s) for PHP.
 .
 PHP (recursive acronym for PHP: Hypertext Preprocessor) is a widely-used
 open source general-purpose scripting language that is especially suited
 for web development and can be embedded into HTML.

IMPORTANT: You need to observe the debian formatting of control file and the lines immediately below the string text ‘Description’ should have a leading blank character. Observe the ensuing snapshot.

Shell

xvi +316 control

We will use sed to replace every string of the form php7.3 by php7.3-zts:

Shell

sed -i -e 's/php7\.3/php7\.3-zts/g' control

We can see the very first line instance modification in the partial side by side comparison snapshot:

Now, we will concatenate the pthreads LICENSE contents into existing debian copyright file, respecting its format, by inserting a blank line followed by the header: License: pthreads

Shell

echo "\nLicense: pthreads" >> copyright

then concatenate the LICENSE file, found in the phthreads directory we cloned and worked on previously:

Shell

cat ../php-7.3.10/ext/pthreads/LICENSE >> copyright

Analyzing the result by opening our modified copyright file with xvi text editor we want to remove those extraneous characters represented as ‘^M‘ which were embedded in pthreads LICENSE. We will use sed and to specify the extraneous characters, I will input CTRLv followed by CTRLm

Shell

sed -i 's/^M//g' copyright

Please see raw concatenated copyright instance to our left and on the right the copyright file operated on by sed:

We continue by integrating our previously created patch metztli-ext-pthreads.patch into debian patches structure. First we list the contents of patches directory:

Shell

ls patches

Then I create a metztli directory and copy metztli-ext-pthreads.patch into it:

Shell

mkdir patches/metztli && cp -iv ../metztli-ext-pthreads.patch patches/metztli/.

finally I include a reference to my patch into the series file which is used by quilt patching utility.

Shell

echo "metztli/metztli-ext-pthreads.patch" >> patches/series

Now I will create a rules.d/pthreads.mk configuration file using xvi text editor:

Shell

xvi rules.d/pthreads.mk

where the content below will be inserted:

Code

ext_PACKAGES      += pthreads
pthreads_DESCRIPTION := KrakJoe pthreads
pthreads_EXTENSIONS  := pthreads
pthreads_PRIORITY    := 10
pthreads_EXTENSION   := krakjoe_extension
pthreads_config      := --enable-pthreads=shared,/usr
export pthreads_EXTENSIONS
export pthreads_DESCRIPTION
export pthreads_PRIORITY
export pthreads_EXTENSION

We save our file and exit xvi

Finally, we are going to hack debian rules file to modify our cli php∙ini file by inserting the proper pthreads directive, CHINGON!

Thus I instantiate a variable, CHINGON, by assigning it the following string of characters to be interpreted by sed:
at line 937 insert ;Metztli IT added required cli directive: (php∙ini comments begin with leading ; semicolon);
then add new line which begins with extension=pthreads and end line with backslash followed by blank character

Shell

CHINGON="-e'937i;Metztli IT added required cli directive:\nextension=pthreads' \ \\"

and use sed to insert $CHINGON value at line 527 in debian rules file (yes, sed will edit in-place but makes a rules.back -up:

Shell

sed -i.back -e 527i$CHINGON rules

And I verify modification in debian rules file by opening it with xvi and editing the inserted text by moving it away from the left edge and aligning, and matching format, with the existing debian code, as below:

I save modifications to debian rules file and exit xvi.

Subsequently, I replace only the very first occurrence of php7∙3 with php7∙3-zts in debian changelog:

Shell

sed -i '0,/php7\.3/s//php7\.3-zts/' changelog

and I am ready to make a patch against Debian packaging for php7.3_7.3.10-1 ZTS and pthreads:

Shell

git add .
git commit -a -m "Enhancing Debian packaging for php7.3_7.3.10-1 to generate ZTS and pthreads versions for CLI"
git format-patch --stdout -1 >../../metztli-php-7.3.10-zts-pthreads-enhanced-cli.patch

metztli-php-7.3.10-zts-pthreads-enhanced-cli.patch
metztli-php-7.3.10-zts-pthreads-enhanced-cli.patch.SHA256SUM

Build Enhanced PHP 7.3.10 by Patching Debian Packaging for php7.3_7.3.10-1.x with ZTS and pthreads.

Having created our Debian packaging for php7.3_7.3.10-1.x, we exit debian directory by going back to our initial directory location –where we downloaded our PHP resources ; our objective is to build actual PHP with ZTS and pthreads hack the ‘Debian way’, hence I create appropriately named directory:

Shell

cd ../.. && mkdir --verbose build-zts-pthreads-real

Again, I untar PHP 7.3.10 source into our newly created directory:

Shell

tar -C build-zts-pthreads-real -xPf php-7.3.10.tar.xz

and untar official debian packaging for php7.3_7.3.10-1 into the PHP 7.3.10 source directory:

Shell

tar -C build-zts-pthreads-real/php-7.3.10 -xPf php7.3_7.3.10-1.debian.tar.xz

I verify that debian directory was untarred at the target location and proceed to change into PHP source directory

Shell

cd build-zts-pthreads-real/php-7.3.10

I check for the existence of pthreads by listing ext/ directory contents… of course, pthreads is not there.

Shell

ls ext/

And test application of my ZTS and pthreads patch into debian packaging for php7.3_7.3.10-1:

Shell

cat ../../metztli-php-7.3.10-zts-pthreads-enhanced-cli.patch | patch -d debian --dry-run --fuzz=0 -p1

and subsequently apply the patch:

Shell

cat ../../metztli-php-7.3.10-zts-pthreads-enhanced-cli.patch | patch -d debian --fuzz=0 -p1

I make a symbolic link to debian/patches directory:

Shell

ln -s debian/patches

and follow with a quilt operation to push…er, apply all patches listed in series file:

Shell

quilt push -a --fuzz=0

we notice that the last patch applied by quilt was our very first metztli-ext-pthreads.patch which adds pthreads source into ext/ directory. We may wish to verify ext/ for the existence of pthreads.
Although not strictly necessary, I do label my builds minimally as follows:

Shell

dch -D metztli

BUT: If we are leveraging Debian packaging for php7.3_7.3.10-1.debian.tar.xz to build upstream PHP 7.3.11 ZTS with pthreads, we need to upgrade the version; otherwise, the generated packages will have lower incorrect version. Thus I need to correct the debian/changelog with:

Shell

dch -v 7.3.11-1 -D metztli

Both 93 and 94 generate a warning because metztli is not an official Debian distribution.

After I dismiss it a text editor opens debian/changelog file for my irreverent annotations. Subsequently, I save my modifications/additions, the editor closes changelog and I am returned to my shell.
The objective of a subset of our modifications was to enable ZTS build, accordingly I type:

Shell

./debian/rules debian/control

Enabling ZTS build — ./debian/rules debian/control

I begin the build procedure, as before, with our tlilxochitl, i.e., ‘vanilla‘, PHP 7.3.10 build:

Shell

dpkg-buildpackage -F -us -uc -jX -T binary-arch,binary-indep

where X implies previous meaning, i.e., Number of jobs allowed to be run simultaneously, number of jobs matching the number of online processors if auto is specified (since dpkg 1.17.10), or unlimited number if jobs is not specified, equivalent to the make(1) option of the same name (since dpkg 1.14.7, long option since dpkg 1.18.8).

Likewise,after thirty(30) to fifty(50) minutes with 6 cores assigned, i.e., -j6, PHP 7.3.10 ZTS and pthreads build procedure ends, generating DEB packages the ‘Debian way’:

NOTE: this build is more recent than the one whose sample output I showed in the cover. Additionally, you need php-common from your favourite debian repository in addition to the first six(6) packages below to try out pthreaded programs.
Remember: pthreads v3 is restricted to operating in CLI only: I have spent many years trying to explain that threads in a web server just don’t make sense, after 1,111 commits to pthreads I have realised that, my advice is going unheeded. ~krakjoe

Corresponding SHA256SUMs:

WARNING!
Although due diligence was exercised, what/whoever entity who decides or is inspired to follow the procedure(s) shown as proof-of-concept (PoC) and/or downloads Metztli Reiser4 for AMD64 -generated PHP 7.3.10/11 ZTS and pthreads does so willingly knowing that I and/or Metztli IT DO NOT OFFER ANY GUARANTEE OR EXTEND ANY WARRANTY ON THESE RESOURCES. Further, I and/or Metztli IT reserve the right to modify the post with corrections and/or additions and/or even delete it without advanced notice.

References:
Вики Одинцова : Viki Odintcova on Instagram.
¹ Ixachitlan great land mass, i.e., ‘continent’, from ixachi: a lot or great in quantity, mass, and tlan: land, earth.
² Xoloitzcuintli, singular; Xoloitzcuintin, plural; Xoloitzcuintzintin, diminutive plural.
³ Their equally stunted descendants who now label themselves as ‘Mexican‘ to continue their plunder unabated, still do.
⁴Xoloitzcuintzintin: ‘almost eaten to extinction by European colonists.’ ~ N.G.
⁵ Halting development of pthreads for 7.4 #929
⁶ Fascinating how certain questionable terms are considered ‘funny‘ and/or ‘creative‘ when used and/or applied by certain demographic group(s) by virtue that their members’ entitlement shields them from potentially undesirable connotations which otherwise besmirch unfortunate members of other group(s) in the universal set of humans.
⁷ Xiuhcohuanahualli: shape shifting and/or transformation into a ‘Fire Serpent‘ ability represented by a device with such likeness carried by Huitzilopochtli on his back.

Compile PHP 7.2 with pthreads
Find and replace text in a file between range of lines using sed
GitHub Gist: build-php-zts.sh
How to use sed to replace only the first occurrence in a file?

Source for threaded Вики∙php

PHP

	`<?php`
	`$thread = new class extends Thread {`
	`public function run() {`
	`echo ":D\n";`
	`echo "\"Как же мало нужно для счастья!\" ~ Виктория Одинцова\n";`
	`echo "...\n";`
	`echo "In Nahuatl, Mexico's genuine language par excellence:\n";`
	`echo "\"Quexquich tepitzin yolcapaquiliztli!\" ~ Victoriyatzin Odintsova\n";`
	`echo "...\n";`
	`echo "Now, in English:\n";`
	`echo "\"How little is needed for happiness!\" ~ Victoriya Odintsova\n";`
	`echo "...\n";`
	`echo "B)\n";`
	`}`
	`};`

	`$thread->start() && $thread->join();`
	`?>`

NOTE: in the above PHP code, I substituted :D, i.e., smiley, at line number 98, and B), i.e, Chingon or sunglasses, at line number 107, respectively, for the code to be saved.

	`Package: php7.3-pthreads`
	`Architecture: any`
	`Depends: php-common (>= 1:35),`
	`ucf,`
	`${misc:Depends},`
	`${php:Depends},`
	`${php-pthreads:Depends},`
	`${shlibs:Depends}`
	`Pre-Depends: ${misc:Pre-Depends},`
	`${php-pthreads:Pre-Depends}`
	`Built-Using: ${php:Built-Using}`
	`Replaces: ${php-pthreads:Replaces}`
	`Breaks: ${php-pthreads:Breaks}`
	`Conflicts: ${php-pthreads:Conflicts}`
	`Provides: ${php-pthreads:Provides}`
	`Recommends: ${php-pthreads:Recommends}`
	`Description: KrakJoe pthreads module for PHP`
	`This package provides the KrakJoe pthreads module(s) for PHP.`
	`.`
	`PHP (recursive acronym for PHP: Hypertext Preprocessor) is a widely-used`
	`open source general-purpose scripting language that is especially suited`
	`for web development and can be embedded into HTML.`

	`ext_PACKAGES += pthreads`
	`pthreads_DESCRIPTION := KrakJoe pthreads`
	`pthreads_EXTENSIONS := pthreads`
	`pthreads_PRIORITY := 10`
	`pthreads_EXTENSION := krakjoe_extension`
	`pthreads_config := --enable-pthreads=shared,/usr`
	`export pthreads_EXTENSIONS`
	`export pthreads_DESCRIPTION`
	`export pthreads_PRIORITY`
	`export pthreads_EXTENSION`

Mon	Tue	Wed	Thu	Fri	Sat	Sun
						1
2	3	4	5	6	7	8
9	10	11	12	13	14	15
16	17	18	19	20	21	22
23	24	25	26	27	28
<< <				> >>

	`wget https://www.php.net/distributions/php-7.3.10.tar.xz https://www.php.net/distributions/php-7.3.10.tar.xz.asc`
	`gpg --verify php-7.3.10.tar.xz.asc php-7.3.10.tar.xz`

	`mkdir --verbose build && tar -C build -xPf php-7.3.10.tar.xz`
	`tar -C build/php-7.3.10 -xPf php7.3_7.3.10-1.debian.tar.xz`

	`git add .`
	`git commit -a -m "Enhancing Debian packaging for php7.3_7.3.10-1 to generate ZTS and pthreads versions for CLI"`
	`git format-patch --stdout -1 >../../metztli-php-7.3.10-zts-pthreads-enhanced-cli.patch`

	`cd ..`
	`ls ..`

Tag: "oaxaca"

Prior to European invasion of Ixachitlan1 there were xoloitzcuintzintin2 –not ‘puppies’…

Build PHP 7.3.10 the ‘Debian way’ on Metztli Reiser4 / Debian 10 Buster

Build PHP 7.3.10 ZTS and pthreads the ‘Debian way’ on Metztli Reiser4

Modifying and simultaneously creating a ZTS and pthreads patch for Debian packaging for php7.3_7.3.10-1 itself.

Build Enhanced PHP 7.3.10 by Patching Debian Packaging for php7.3_7.3.10-1.x with ZTS and pthreads.

Prior to European invasion of Ixachitlan¹ there were xoloitzcuintzintin² –not ‘puppies’…