サイドバー
memo:2019:08:2019-08-17
tweeper
$ apt show tweeper Package: tweeper Version: 1.4.0-1 Priority: optional Section: php Maintainer: Antonio Ospite <ao2@ao2.it> Installed-Size: 89.1 kB Depends: php-cli, php-common, php-curl, php-xml, php-json, php-symfony-serializer (>= 2.7.0), php-symfony-property-access (>= 2.7.0) Suggests: libapache2-mod-php | php-cgi Homepage: https://git.ao2.it/tweeper.git Download-Size: 19.2 kB APT-Manual-Installed: yes APT-Sources: http://ftp.jp.debian.org/debian sid/main amd64 Packages Description: web scraper to convert supported websites like Twitter.com to RSS tweeper is a web scraper which can be used to conveniently follow the public activity of social network users without the need to log in or even be subscribed to the social network; tweeper converts the public information to RSS so that it can be accessed and collected by a feed reader. . tweeper started as the TWitter fEEd scraPER but support for other web sites has been added. . The sites that tweeper is able to scrape and convert to RSS are: . * Twitter.com * Pump.io based websites, like Identi.ca * Dilbert.com * Instagram.com * Facebook.com (public pages) . tweeper can be used as: . 1. a command line tool; 2. a filter for feed readers; 3. a web based tool when used with a PHP-enabled web server.
install
$ sudo apt install tweeper
man
TWEEPER(1) TWEEPER(1)
NAME
tweeper - web scraper to convert supported websites (e.g. Twitter.com) to RSS
SYNOPSIS
tweeper [OPTIONS] URL
DESCRIPTION
tweeper(1) is a web scraper which can be used to conveniently follow the public activity of social network users without the need to log in or even be subscribed to
the social network; tweeper converts the public information to RSS so that it can be accessed and collected by a feed reader.
tweeper started as the TWitter fEEd scraPER but support for other web sites has been added.
The sites that tweeper is able to scrape and convert to RSS are:
• Twitter.com
• Pump.io based websites, like Identi.ca
• Dilbert.com
• Instagram.com
• Facebook.com (public pages)
tweeper can be used as:
1. a command line tool;
2. a filter for feed readers;
3. a web based tool when used with a PHP-enabled web server.
OPTIONS
-e
show links to supported media files in the RSS <enclosure/> element
-m <0|1>
enable or disable showing multimedia content (e.g. Twitter or Instagram pictures) directly inside the item description. Default is 1 (enable).
-u <0|1>
enable or disable showing usernames in front of the item for hosts which supports it (Twitter.com/Instagram.com). Default is 1 (enable).
-h, --help
show the help message
EXAMPLE OF USE
Getting the RSS feed of some Twitter user:
tweeper https://twitter.com/NSACareers
Using tweeper as a filter for the Liferea feed reader:
liferea-add-feed "|tweeper https://twitter.com/NSAcareers"
To use tweeper via web there are two options (the examples assume the installation directory to be /usr/share/php/tweeper/):
1. Using the PHP built-in web server:
php -S localhost:8000 -t /usr/share/php/tweeper/
and then visit http://localhost:8000/tweeper.php in the web browser.
2. Using a generic web server with the document root in /var/www:
sudo ln -s /usr/share/php/tweeper/tweeper.php /var/www
xdg-open http://localhost/tweeper.php?src_url=http://twitter.com/NSAcareers
It is enough to create the symlink only the very first time tweeper is used this way.
NOTES
In order to use tweeper with a symlink with the apache userdir module, the SymLinksIfOwnerMatch option must be replaced by FollowSymlink in
/etc/apache2/mods-enabled/userdir.conf
EXIT STATUS
0
Success
!0
Failure
AUTHORS
Antonio Ospite
RESOURCES
Main web site: https://git.ao2.it/tweeper.git
COPYING
Copyright (C) 2013-2018 Antonio Ospite <ao2@ao2.it>
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software
Foundation, either version 3 of the License, or (at your option) any later version.
11/17/2018 TWEEPER(1)
twitterをRSSに変換
$ tweeper https://twitter.com/matoken | xmllint - | head
PHP Warning: Error 513: ID content-main-heading already defined in Entity, line 1281 in /usr/share/php/tweeper/src/Tweeper.php on line 256
PHP Warning: Error 513: ID saved-searches-heading already defined in Entity, line 7398 in /usr/share/php/tweeper/src/Tweeper.php on line 256
<?xml version="1.0"?>
<rss version="2.0" xml:base="https://twitter.com">
<channel>
<generator>Tweeper</generator>
<title>Twitter / matoken</title>
<link>https://twitter.com/matoken</link>
<description>@matoken@tw @matoken Google+が無くなって途方に暮れている無職の自転車乗り http://gnusocial.matoken.org/matoken  がメイン feedは邪魔という方がいたので@matokenfで流すようにしました 秋葉原に住みたい地方民 matoken.org Joined April 2007</description>
<image>
<title>Twitter / matoken</title>
<link>https://twitter.com/matoken</link>
facebookをRSSに
$ tweeper https://www.facebook.com/matoken | xmllint - | head
PHP Warning: Error 42: Attribute data-referrer redefined in Entity, line 12 in /usr/share/php/tweeper/src/Tweeper.php on line 256
PHP Warning: Error 42: Attribute data-referrer redefined in Entity, line 12 in /usr/share/php/tweeper/src/Tweeper.php on line 256
PHP Warning: Error 42: Attribute data-referrer redefined in Entity, line 12 in /usr/share/php/tweeper/src/Tweeper.php on line 256
PHP Warning: Error 42: Attribute data-referrer redefined in Entity, line 12 in /usr/share/php/tweeper/src/Tweeper.php on line 256
PHP Warning: Error 42: Attribute data-referrer redefined in Entity, line 12 in /usr/share/php/tweeper/src/Tweeper.php on line 256
PHP Warning: Error 42: Attribute data-referrer redefined in Entity, line 12 in /usr/share/php/tweeper/src/Tweeper.php on line 256
PHP Warning: Error 42: Attribute data-referrer redefined in Entity, line 12 in /usr/share/php/tweeper/src/Tweeper.php on line 256
PHP Warning: Error 42: Attribute data-referrer redefined in Entity, line 12 in /usr/share/php/tweeper/src/Tweeper.php on line 256
PHP Warning: Error 42: Attribute data-referrer redefined in Entity, line 12 in /usr/share/php/tweeper/src/Tweeper.php on line 256
PHP Warning: Error 42: Attribute data-referrer redefined in Entity, line 12 in /usr/share/php/tweeper/src/Tweeper.php on line 256
PHP Warning: Error 42: Attribute data-referrer redefined in Entity, line 12 in /usr/share/php/tweeper/src/Tweeper.php on line 256
PHP Warning: Error 42: Attribute data-referrer redefined in Entity, line 12 in /usr/share/php/tweeper/src/Tweeper.php on line 256
PHP Warning: Error 42: Attribute data-referrer redefined in Entity, line 12 in /usr/share/php/tweeper/src/Tweeper.php on line 256
PHP Warning: Error 42: Attribute data-referrer redefined in Entity, line 12 in /usr/share/php/tweeper/src/Tweeper.php on line 256
PHP Warning: Error 801: Tag wbr invalid in Entity, line 12 in /usr/share/php/tweeper/src/Tweeper.php on line 256
PHP Warning: Error 801: Tag wbr invalid in Entity, line 12 in /usr/share/php/tweeper/src/Tweeper.php on line 256
<?xml version="1.0"?>
<rss version="2.0" xml:base="https://facebook.com">
<channel>
<generator>Tweeper</generator>
<title>Kenichiro Matohara | Facebook</title>
<link>https://www.facebook.com/matoken</link>
<description><![CDATA[Kenichiro Matohara is on Facebook. Join Facebook to connect with Kenichiro Matohara and others you may know. Facebook gives people the power to share and makes the world more open and connected.]]></description>
<image>
<title>Kenichiro Matohara | Facebook</title>
<link>https://www.facebook.com/matoken</link>
instagramを
$ tweeper https://www.instagram.com/matoken | xmllint - | head
<?xml version="1.0"?>
<rss version="2.0" xml:base="https://instagram.com">
<channel>
<generator>Tweeper</generator>
<title>Instagram / matoken</title>
<link>https://instagram.com/matoken</link>
<description><![CDATA[matoken. 猫と過去多め <a href="https://matoken.org/">https://matoken.org/</a>]]></description>
<image>
<title>Instagram / matoken</title>
<link>https://instagram.com/matoken</link>
phpビルトインWebServer経由で起動
$ php -S localhost:8000 -t /usr/share/php/tweeper/
起動したtweeperにアクセス
$ w3m -dump http://localhost:8000/tweeper.php usage: /tweeper.php?src_url=<src_url>&generate_enclosure=<0|1>&show_usernames= <0|1>&show_multimedia=<0|1>
Web経由でRSSに変換
$ w3m -dump_source http://localhost:8000/tweeper.php?src_url=https://twitter.com/matoken | head
<?xml version="1.0"?>
<rss version="2.0" xml:base="https://twitter.com">
<channel>
<generator>Tweeper</generator>
<title>Twitter / matoken</title>
<link>https://twitter.com/matoken</link>
<description>@matoken@tw @matoken Google+が無くなって途方に暮れている無職の自転車乗り http://gnusocial.matoken.org/matoken がメイン feedは邪魔という方がいたので@matokenfで
流すようにしました 秋葉原に住みたい地方民 matoken.org Joined April 2007</description>
<image>
<title>Twitter / matoken</title>
<link>https://twitter.com/matoken</link>
memo/2019/08/2019-08-17.txt · 最終更新: 2019/08/17 12:14 by matoken
特に明示されていない限り、本Wikiの内容は次のライセンスに従います： CC Attribution-Noncommercial-Share Alike 4.0 International
コメント