Python 3 auf Mac OS installieren

Python 3 Installation in drei einfachen Schritten und in unter 5 Minuten

Der Auslöser Python 3 auf meinem Mac zu installieren war, dass ich gerne den Webcrawler Scrapy ausprobieren möchte. Dieser kann mir bei einem aktuellen Projekt vielleicht wertvolle Dienste leisten. Daneben gibt es noch 1000 weitere gute Gründe Python 3 auf dem Rechner zu installieren. Für mich ist es, dass ich mich in einem nächsten Schritt stärker mit Neuronalen Netzen beschäftigen möchte. Eine der besten Sprachen um dies zu tun ist wiederum Python. Hier gibt es einige tolle Frameworks, die einem die Arbeit erleichtern und damit auch den Einstieg in die Materie. Dazu jedoch mehr, wenn ich tatsächlich am experimentieren bin.

Voraussetzung:

  • Homebrew ist für Mac OS installiert – folgt dazu der Anleitung auf https://brew.sh

1) Python installieren

HS-MacBook-Pro:~ hstridde$ brew install python

Die Ausgabe kann dann wie folgt aussehen:

Updating Homebrew...
==> Auto-updated Homebrew!
Updated 1 tap (homebrew/core).
==> Updated Formulae
libgda

==> Installing dependencies for python: gdbm, openssl, readline, sqlite and xz
==> Installing python dependency: gdbm
==> Downloading https://homebrew.bintray.com/bottles/gdbm-1.18.1.mojave.bottle.1.tar.gz
######################################################################## 100.0%
==> Pouring gdbm-1.18.1.mojave.bottle.1.tar.gz
🍺  /usr/local/Cellar/gdbm/1.18.1: 20 files, 586.8KB
==> Installing python dependency: openssl
==> Downloading https://homebrew.bintray.com/bottles/openssl-1.0.2q.mojave.bottle.tar.gz
######################################################################## 100.0%
==> Pouring openssl-1.0.2q.mojave.bottle.tar.gz
==> Caveats
A CA file has been bootstrapped using certificates from the SystemRoots
keychain. To add additional certificates (e.g. the certificates added in
the System keychain), place .pem files in
  /usr/local/etc/openssl/certs

and run
  /usr/local/opt/openssl/bin/c_rehash

openssl is keg-only, which means it was not symlinked into /usr/local,
because Apple has deprecated use of OpenSSL in favor of its own TLS and crypto libraries.

If you need to have openssl first in your PATH run:
  echo 'export PATH="/usr/local/opt/openssl/bin:$PATH"' >> ~/.bash_profile

For compilers to find openssl you may need to set:
  export LDFLAGS="-L/usr/local/opt/openssl/lib"
  export CPPFLAGS="-I/usr/local/opt/openssl/include"

==> Summary
🍺  /usr/local/Cellar/openssl/1.0.2q: 1,794 files, 12.1MB
==> Installing python dependency: readline
==> Downloading https://homebrew.bintray.com/bottles/readline-8.0.0.mojave.bottle.tar.gz
######################################################################## 100.0%
==> Pouring readline-8.0.0.mojave.bottle.tar.gz
==> Caveats
readline is keg-only, which means it was not symlinked into /usr/local,
because macOS provides the BSD libedit library, which shadows libreadline.
In order to prevent conflicts when programs look for libreadline we are
defaulting this GNU Readline installation to keg-only.

For compilers to find readline you may need to set:
  export LDFLAGS="-L/usr/local/opt/readline/lib"
  export CPPFLAGS="-I/usr/local/opt/readline/include"

==> Summary
🍺  /usr/local/Cellar/readline/8.0.0: 48 files, 1.5MB
==> Installing python dependency: sqlite
==> Downloading https://homebrew.bintray.com/bottles/sqlite-3.26.0_1.mojave.bottle.1.tar.gz
######################################################################## 100.0%
==> Pouring sqlite-3.26.0_1.mojave.bottle.1.tar.gz
==> Caveats
sqlite is keg-only, which means it was not symlinked into /usr/local,
because macOS provides an older sqlite3.

If you need to have sqlite first in your PATH run:
  echo 'export PATH="/usr/local/opt/sqlite/bin:$PATH"' >> ~/.bash_profile

For compilers to find sqlite you may need to set:
  export LDFLAGS="-L/usr/local/opt/sqlite/lib"
  export CPPFLAGS="-I/usr/local/opt/sqlite/include"

==> Summary
🍺  /usr/local/Cellar/sqlite/3.26.0_1: 11 files, 3.7MB
==> Installing python dependency: xz
==> Downloading https://homebrew.bintray.com/bottles/xz-5.2.4.mojave.bottle.tar.gz
######################################################################## 100.0%
==> Pouring xz-5.2.4.mojave.bottle.tar.gz
🍺  /usr/local/Cellar/xz/5.2.4: 92 files, 1MB
==> Installing python
==> Downloading https://homebrew.bintray.com/bottles/python-3.7.2_1.mojave.bottle.1.tar.gz
######################################################################## 100.0%
==> Pouring python-3.7.2_1.mojave.bottle.1.tar.gz
==> /usr/local/Cellar/python/3.7.2_1/bin/python3 -s setup.py --no-user-cfg install --force --verbose --install-scripts=/usr/local/Cellar/python/3.7.
==> /usr/local/Cellar/python/3.7.2_1/bin/python3 -s setup.py --no-user-cfg install --force --verbose --install-scripts=/usr/local/Cellar/python/3.7.
==> /usr/local/Cellar/python/3.7.2_1/bin/python3 -s setup.py --no-user-cfg install --force --verbose --install-scripts=/usr/local/Cellar/python/3.7.
==> Caveats
Python has been installed as
  /usr/local/bin/python3

Unversioned symlinks `python`, `python-config`, `pip` etc. pointing to
`python3`, `python3-config`, `pip3` etc., respectively, have been installed into
  /usr/local/opt/python/libexec/bin

If you need Homebrew's Python 2.7 run
  brew install [email protected]

You can install Python packages with
  pip3 install <package>
They will install into the site-package directory
  /usr/local/lib/python3.7/site-packages

See: https://docs.brew.sh/Homebrew-and-Python
==> Summary
🍺  /usr/local/Cellar/python/3.7.2_1: 3,833 files, 59.3MB
==> Caveats
==> openssl
A CA file has been bootstrapped using certificates from the SystemRoots
keychain. To add additional certificates (e.g. the certificates added in
the System keychain), place .pem files in
  /usr/local/etc/openssl/certs

and run
  /usr/local/opt/openssl/bin/c_rehash

openssl is keg-only, which means it was not symlinked into /usr/local,
because Apple has deprecated use of OpenSSL in favor of its own TLS and crypto libraries.

If you need to have openssl first in your PATH run:
  echo 'export PATH="/usr/local/opt/openssl/bin:$PATH"' >> ~/.bash_profile

For compilers to find openssl you may need to set:
  export LDFLAGS="-L/usr/local/opt/openssl/lib"
  export CPPFLAGS="-I/usr/local/opt/openssl/include"

==> readline
readline is keg-only, which means it was not symlinked into /usr/local,
because macOS provides the BSD libedit library, which shadows libreadline.
In order to prevent conflicts when programs look for libreadline we are
defaulting this GNU Readline installation to keg-only.

For compilers to find readline you may need to set:
  export LDFLAGS="-L/usr/local/opt/readline/lib"
  export CPPFLAGS="-I/usr/local/opt/readline/include"

==> sqlite
sqlite is keg-only, which means it was not symlinked into /usr/local,
because macOS provides an older sqlite3.

If you need to have sqlite first in your PATH run:
  echo 'export PATH="/usr/local/opt/sqlite/bin:$PATH"' >> ~/.bash_profile

For compilers to find sqlite you may need to set:
  export LDFLAGS="-L/usr/local/opt/sqlite/lib"
  export CPPFLAGS="-I/usr/local/opt/sqlite/include"

==> python
Python has been installed as
  /usr/local/bin/python3

Unversioned symlinks `python`, `python-config`, `pip` etc. pointing to
`python3`, `python3-config`, `pip3` etc., respectively, have been installed into
  /usr/local/opt/python/libexec/bin

If you need Homebrew's Python 2.7 run
  brew install [email protected]

You can install Python packages with
  pip3 install <package>
They will install into the site-package directory
  /usr/local/lib/python3.7/site-packages

See: https://docs.brew.sh/Homebrew-and-Python

2) Installation prüfen

Bitte prüfe, dass alles auf dem neusten Stand ist:

$ brew update; brew upgrade python

Bei mir kommt relativ wenig, da ich Homebrew erst vor kurzem upgedated habe. Ein Hinweis, Homebrew ist erst einmal nicht sehr gesprächig. Bei einem Update nach längerer Zeit ist über eine Minute auf der Shell nichts passiert, bis dann eine Ausgabe kam. Hier einfach die Ruhe bewahren. Wenn ihr es gerade frisch installiert habt, wird es sicher ähnlich aussehen:

Updated 1 tap (homebrew/core).
==> Updated Formulae
wine
Error: python 3.7.2_1 already installed

Nun lassen wir uns noch einmal anzeigen welche Python Version wir installiert haben:

$ python --version
Python 2.7.10

Was ist denn hier los? Alles ist in Ordnung. Um zu sehen in welcher Version python 3 vorliegt, müssen wir auch Python 3 aufrufen. Das geht wie folgt:

$ python3 --version
Python 3.7.2

Schon sieht die Ausgabe deutlich besser aus und stimmt mit dem überein, was uns auch der Installer ausgegeben habt.

Nachdem du python 3 installiert hast, prüfe bitte ob es funktionsbereit ist indem du die interaktive Konsole von Python 3 aufrufst.

$ python3

Wenn alles klappt siehst du das folgende:

Python 3.7.2 (default, Jan 13 2019, 12:50:01)
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type „help“, „copyright“, „credits“ or „license“ for more information.

Um den Interpreter zu verlasen drücke nun STRG+D.

3) Scrapy installieren

Der Grund für die Python 3 Installation war, dass ich das den Webcrawler Scrapy ausprobieren möchte. Also nutzen wir nun das zugehörige Paket mit pip. Auch hier der Hinweis:

  • pip nutzt Python 2.7
  • pip3 nutzt Python 3.7

Wenn du ein Paket für beide Python Versionen nutzen möchtest, so musst du es auch für beide Versionen separat installieren. Einmal mit pip für Version 2.7 und einmal mit pip3 für Version 3.7

Für die Installation von Scapy bedeutet dies:

$ pip3 install Scrapy

Es werden etliche Pakete als Abhängigkeit zu Scrapy installiert. Die Ausgabe kann wie folgt aussehen:

Collecting Scrapy
  Downloading https://files.pythonhosted.org/packages/af/23/33b950071c335714e550a982dd380e009f313659e4a3fa3f3dc4dcb77fee/Scrapy-1.5.2-py2.py3-none-any.whl (251kB)
    100% |████████████████████████████████| 256kB 6.4MB/s
Collecting PyDispatcher>=2.0.5 (from Scrapy)
  Downloading https://files.pythonhosted.org/packages/cd/37/39aca520918ce1935bea9c356bcbb7ed7e52ad4e31bff9b943dfc8e7115b/PyDispatcher-2.0.5.tar.gz
Collecting w3lib>=1.17.0 (from Scrapy)
  Downloading https://files.pythonhosted.org/packages/81/43/9dcf92a77f5f0afe4f4df2407d7289eea01368a08b64bda00dd318ca62a6/w3lib-1.20.0-py2.py3-none-any.whl
Collecting lxml (from Scrapy)
  Downloading https://files.pythonhosted.org/packages/8b/fb/f56f79fd3ab36774a5d8662deb0871cce815c8f8a1994ede49b30e678741/lxml-4.3.0-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (8.7MB)
    100% |████████████████████████████████| 8.7MB 3.8MB/s
Collecting pyOpenSSL (from Scrapy)
  Downloading https://files.pythonhosted.org/packages/01/c8/ceb170d81bd3941cbeb9940fc6cc2ef2ca4288d0ca8929ea4db5905d904d/pyOpenSSL-19.0.0-py2.py3-none-any.whl (53kB)
    100% |████████████████████████████████| 61kB 17.0MB/s
Collecting service-identity (from Scrapy)
  Downloading https://files.pythonhosted.org/packages/e9/7c/2195b890023e098f9618d43ebc337d83c8b38d414326685339eb024db2f6/service_identity-18.1.0-py2.py3-none-any.whl
Collecting parsel>=1.1 (from Scrapy)
  Downloading https://files.pythonhosted.org/packages/96/69/d1d5dba5e4fecd41ffd71345863ed36a45975812c06ba77798fc15db6a64/parsel-1.5.1-py2.py3-none-any.whl
Collecting Twisted>=13.1.0 (from Scrapy)
  Downloading https://files.pythonhosted.org/packages/5d/0e/a72d85a55761c2c3ff1cb968143a2fd5f360220779ed90e0fadf4106d4f2/Twisted-18.9.0.tar.bz2 (3.1MB)
    100% |████████████████████████████████| 3.1MB 10.0MB/s
Collecting six>=1.5.2 (from Scrapy)
  Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting cssselect>=0.9 (from Scrapy)
  Downloading https://files.pythonhosted.org/packages/7b/44/25b7283e50585f0b4156960691d951b05d061abf4a714078393e51929b30/cssselect-1.0.3-py2.py3-none-any.whl
Collecting queuelib (from Scrapy)
  Downloading https://files.pythonhosted.org/packages/4c/85/ae64e9145f39dd6d14f8af3fa809a270ef3729f3b90b3c0cf5aa242ab0d4/queuelib-1.5.0-py2.py3-none-any.whl
Collecting cryptography>=2.3 (from pyOpenSSL->Scrapy)
  Downloading https://files.pythonhosted.org/packages/d7/9e/12bb10fd009b0146935c169cc0e1e86221eacf8dc207990d54b783c47a7d/cryptography-2.5-cp34-abi3-macosx_10_6_intel.whl (1.7MB)
    100% |████████████████████████████████| 1.7MB 14.2MB/s
Collecting pyasn1 (from service-identity->Scrapy)
  Downloading https://files.pythonhosted.org/packages/7b/7c/c9386b82a25115cccf1903441bba3cbadcfae7b678a20167347fa8ded34c/pyasn1-0.4.5-py2.py3-none-any.whl (73kB)
    100% |████████████████████████████████| 81kB 22.8MB/s
Collecting pyasn1-modules (from service-identity->Scrapy)
  Downloading https://files.pythonhosted.org/packages/da/98/8ddd9fa4d84065926832bcf2255a2b69f1d03330aa4d1c49cc7317ac888e/pyasn1_modules-0.2.4-py2.py3-none-any.whl (66kB)
    100% |████████████████████████████████| 71kB 18.8MB/s
Collecting attrs>=16.0.0 (from service-identity->Scrapy)
  Downloading https://files.pythonhosted.org/packages/3a/e1/5f9023cc983f1a628a8c2fd051ad19e76ff7b142a0faf329336f9a62a514/attrs-18.2.0-py2.py3-none-any.whl
Collecting zope.interface>=4.4.2 (from Twisted>=13.1.0->Scrapy)
  Downloading https://files.pythonhosted.org/packages/d9/3a/101934e0f2026f0a58698978bfedec6e2021b28b846d9e1d9b54369e044d/zope.interface-4.6.0-cp37-cp37m-macosx_10_14_x86_64.whl (131kB)
    100% |████████████████████████████████| 133kB 33.6MB/s
Collecting constantly>=15.1 (from Twisted>=13.1.0->Scrapy)
  Downloading https://files.pythonhosted.org/packages/b9/65/48c1909d0c0aeae6c10213340ce682db01b48ea900a7d9fce7a7910ff318/constantly-15.1.0-py2.py3-none-any.whl
Collecting incremental>=16.10.1 (from Twisted>=13.1.0->Scrapy)
  Downloading https://files.pythonhosted.org/packages/f5/1d/c98a587dc06e107115cf4a58b49de20b19222c83d75335a192052af4c4b7/incremental-17.5.0-py2.py3-none-any.whl
Collecting Automat>=0.3.0 (from Twisted>=13.1.0->Scrapy)
  Downloading https://files.pythonhosted.org/packages/a3/86/14c16bb98a5a3542ed8fed5d74fb064a902de3bdd98d6584b34553353c45/Automat-0.7.0-py2.py3-none-any.whl
Collecting hyperlink>=17.1.1 (from Twisted>=13.1.0->Scrapy)
  Downloading https://files.pythonhosted.org/packages/a7/b6/84d0c863ff81e8e7de87cff3bd8fd8f1054c227ce09af1b679a8b17a9274/hyperlink-18.0.0-py2.py3-none-any.whl
Collecting PyHamcrest>=1.9.0 (from Twisted>=13.1.0->Scrapy)
  Downloading https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl (52kB)
    100% |████████████████████████████████| 61kB 18.8MB/s
Collecting asn1crypto>=0.21.0 (from cryptography>=2.3->pyOpenSSL->Scrapy)
  Downloading https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl (101kB)
    100% |████████████████████████████████| 102kB 21.9MB/s
Collecting cffi!=1.11.3,>=1.8 (from cryptography>=2.3->pyOpenSSL->Scrapy)
  Downloading https://files.pythonhosted.org/packages/0b/ba/32835c9965d8a0090723e1d0b47373365525c4bd08c807b5efdc9fecbc99/cffi-1.11.5-cp37-cp37m-macosx_10_9_x86_64.whl (163kB)
    100% |████████████████████████████████| 163kB 20.1MB/s
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/site-packages (from zope.interface>=4.4.2->Twisted>=13.1.0->Scrapy) (40.6.3)
Collecting idna>=2.5 (from hyperlink>=17.1.1->Twisted>=13.1.0->Scrapy)
  Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB)
    100% |████████████████████████████████| 61kB 17.8MB/s
Collecting pycparser (from cffi!=1.11.3,>=1.8->cryptography>=2.3->pyOpenSSL->Scrapy)
  Downloading https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz (158kB)
    100% |████████████████████████████████| 163kB 28.8MB/s
Building wheels for collected packages: PyDispatcher, Twisted, pycparser
  Running setup.py bdist_wheel for PyDispatcher ... done
  Stored in directory: /Users/MYUSER/Library/Caches/pip/wheels/88/99/96/cfef6665f9cb1522ee6757ae5955feedf2fe25f1737f91fa7f
  Running setup.py bdist_wheel for Twisted ... done
  Stored in directory: /Users/MYUSER/Library/Caches/pip/wheels/57/2e/89/11ba83bc08ac30a5e3a6005f0310c78d231b96a270def88ca0
  Running setup.py bdist_wheel for pycparser ... done
  Stored in directory: /Users/MYUSER/Library/Caches/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511
Successfully built PyDispatcher Twisted pycparser
Installing collected packages: PyDispatcher, six, w3lib, lxml, asn1crypto, pycparser, cffi, cryptography, pyOpenSSL, pyasn1, pyasn1-modules, attrs, service-identity, cssselect, parsel, zope.interface, constantly, incremental, Automat, idna, hyperlink, PyHamcrest, Twisted, queuelib, Scrapy
Successfully installed Automat-0.7.0 PyDispatcher-2.0.5 PyHamcrest-1.9.0 Scrapy-1.5.2 Twisted-18.9.0 asn1crypto-0.24.0 attrs-18.2.0 cffi-1.11.5 constantly-15.1.0 cryptography-2.5 cssselect-1.0.3 hyperlink-18.0.0 idna-2.8 incremental-17.5.0 lxml-4.3.0 parsel-1.5.1 pyOpenSSL-19.0.0 pyasn1-0.4.5 pyasn1-modules-0.2.4 pycparser-2.19 queuelib-1.5.0 service-identity-18.1.0 six-1.12.0 w3lib-1.20.0 zope.interface-4.6.0

Auf der Homepage von Scrapy wird davor gewarnt, dass es zu Problemen auf Mac OS kommen kann aufgrund der Abhängigkeiten bei den kryptografischen Paketen kommen kann. Da dies hier nicht passiert ist, verweise kann ich nur auf die Dokumentation von Scrapy verweisen. Hier nur der Test, dass wir sehen, dass es reibungslos funktioniert. Um zu zeigen, dass die Installation erfolgreich zeige ich direkt den Befehl und die Ausgabe:

$ scrapy bench


2019-01-30 11:43:31 [scrapy.utils.log] INFO: Scrapy 1.5.2 started (bot: scrapybot)
2019-01-30 11:43:31 [scrapy.utils.log] INFO: Versions: lxml 4.3.0.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 18.9.0, Python 3.7.2 (default, Jan 13 2019, 12:50:01) - [Clang 10.0.0 (clang-1000.11.45.5)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1a  20 Nov 2018), cryptography 2.5, Platform Darwin-18.2.0-x86_64-i386-64bit
2019-01-30 11:43:33 [scrapy.crawler] INFO: Overridden settings: {'CLOSESPIDER_TIMEOUT': 10, 'LOGSTATS_INTERVAL': 1, 'LOG_LEVEL': 'INFO'}
2019-01-30 11:43:33 [scrapy.extensions.telnet] INFO: Telnet Password: ec9b1f9ec528e02d
2019-01-30 11:43:33 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.memusage.MemoryUsage',
 'scrapy.extensions.closespider.CloseSpider',
 'scrapy.extensions.logstats.LogStats']
2019-01-30 11:43:33 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
 'scrapy.downloadermiddlewares.stats.DownloaderStats']
2019-01-30 11:43:33 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware']
2019-01-30 11:43:33 [scrapy.middleware] INFO: Enabled item pipelines:
[]
2019-01-30 11:43:33 [scrapy.core.engine] INFO: Spider opened
2019-01-30 11:43:34 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2019-01-30 11:43:35 [scrapy.extensions.logstats] INFO: Crawled 69 pages (at 4140 pages/min), scraped 0 items (at 0 items/min)
2019-01-30 11:43:36 [scrapy.extensions.logstats] INFO: Crawled 125 pages (at 3360 pages/min), scraped 0 items (at 0 items/min)
2019-01-30 11:43:37 [scrapy.extensions.logstats] INFO: Crawled 189 pages (at 3840 pages/min), scraped 0 items (at 0 items/min)
2019-01-30 11:43:38 [scrapy.extensions.logstats] INFO: Crawled 245 pages (at 3360 pages/min), scraped 0 items (at 0 items/min)
2019-01-30 11:43:39 [scrapy.extensions.logstats] INFO: Crawled 293 pages (at 2880 pages/min), scraped 0 items (at 0 items/min)
2019-01-30 11:43:40 [scrapy.extensions.logstats] INFO: Crawled 341 pages (at 2880 pages/min), scraped 0 items (at 0 items/min)
2019-01-30 11:43:41 [scrapy.extensions.logstats] INFO: Crawled 389 pages (at 2880 pages/min), scraped 0 items (at 0 items/min)
2019-01-30 11:43:42 [scrapy.extensions.logstats] INFO: Crawled 429 pages (at 2400 pages/min), scraped 0 items (at 0 items/min)
2019-01-30 11:43:43 [scrapy.extensions.logstats] INFO: Crawled 477 pages (at 2880 pages/min), scraped 0 items (at 0 items/min)
2019-01-30 11:43:44 [scrapy.core.engine] INFO: Closing spider (closespider_timeout)
2019-01-30 11:43:44 [scrapy.extensions.logstats] INFO: Crawled 525 pages (at 2880 pages/min), scraped 0 items (at 0 items/min)
2019-01-30 11:43:44 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 233328,
 'downloader/request_count': 541,
 'downloader/request_method_count/GET': 541,
 'downloader/response_bytes': 1583777,
 'downloader/response_count': 541,
 'downloader/response_status_count/200': 541,
 'finish_reason': 'closespider_timeout',
 'finish_time': datetime.datetime(2019, 1, 30, 10, 43, 44, 911933),
 'log_count/INFO': 18,
 'memusage/max': 52256768,
 'memusage/startup': 52256768,
 'request_depth_max': 19,
 'response_received_count': 541,
 'scheduler/dequeued': 541,
 'scheduler/dequeued/memory': 541,
 'scheduler/enqueued': 10818,
 'scheduler/enqueued/memory': 10818,
 'start_time': datetime.datetime(2019, 1, 30, 10, 43, 34, 17538)}
2019-01-30 11:43:44 [scrapy.core.engine] INFO: Spider closed (closespider_timeout)

Damit ist die Installation erfolgreich beendet und das experimentieren mit Scrapy kann beginnen.

Leave a Comment

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind mit * markiert.