[fanfix.git] / README.md

English [Français](README-fr.md)

# Fanfix

Fanfix is a small Java program that can download stories from some supported websites and render them offline.

## 🔴 This is the command line and server program

You can also use:
- the graphical client [Fanfix-swing](https://github.com/nikiroo/fanfix-swing/)
- the TUI client [Fanfix-jexer](https://github.com/nikiroo/fanfix-jexer/)

## Synopsis

- ```fanfix``` --import [*URL*]
- ```fanfix``` --export [*id*] [*output_type*] [*target*]
- ```fanfix``` --convert [*URL*] [*output_type*] [*target*] (+info)
- ```fanfix``` --read [*id*] ([*chapter number*])
- ```fanfix``` --read-url [*URL*] ([*chapter number*])
- ```fanfix``` --search
- ```fanfix``` --search [*where*] [*keywords*] (page [*page*]) (item [*item*])
- ```fanfix``` --search-tag
- ```fanfix``` --search-tag [*index 1*]... (page [*page*]) (item [*item*])
- ```fanfix``` --list
- ```fanfix``` --server [*key*] [*port*]
- ```fanfix``` --stop-server [*key*] [*port*]
- ```fanfix``` --remote [*key*] [*host*] [*port*]
- ```fanfix``` --help

## Description

(If you are interested in the recent changes, please check the [Changelog](changelog.md) -- note that starting from version 1.4.0, the changelog is checked at startup.)

This program will convert from a (supported) URL to an .epub file for stories or a .cbz file for comics (a few other output types are also available, like Plain Text, LaTeX, HTML...).

To help organize your stories, it can also work as a local library so you can:

- Import a story from its URL (or just from a file)
- Export a story to a file (in any of the supported output types)
- Display a story from the local library in text format in the console
- Via [fanfix-swing](https://github.com/nikiroo/fanfix-swing/): Display a story from the local library graphically **by itself** ([fanfix-swing](https://github.com/nikiroo/fanfix-swing/)) or **by calling a native program to handle it** (potentially converted into HTML before hand, so any browser can open it)

### Supported websites

Currently, the following websites are supported:

- http://FimFiction.net/: fan fictions devoted to the My Little Pony show
- http://Fanfiction.net/: fan fictions of many, many different universes, from TV shows to novels to games
- http://mangahub.io/: a well filled repository of mangas (English)
- https://e621.net/: a Furry website supporting comics, including MLP
- https://sofurry.com/: same thing, but story-oriented
- https://e-hentai.org/: done upon request (so, feel free to ask for more websites!)
- http://mangas-lecture-en-ligne.fr/: a website offering a lot of mangas (in French)

### Support file types

We support a few file types for local story conversion (both as input and as output):

- epub: .epub files created by this program (we do not support "all" .epub files, at least for now)
- text: local stories encoded in plain text format, with a few specific rules:
	- the title must be on the first line
	- the author (preceded by nothing, ```by ``` or ```©```) must be on the second line, possibly with the publication date in parenthesis (i.e., ```By Unknown (3rd October 1998)```)
	- chapters must be declared with ```Chapter x``` or ```Chapter x: NAME OF THE CHAPTER```, where ```x``` is the chapter number
	- a description of the story must be given as chapter number 0
	- a cover image may be present with the same filename as the story, but a .png, .jpeg or .jpg extension
- info_text: contains the same information as the text format, but with a companion .info file to store some metadata (the .info file is supposed to be created by Fanfix or compatible with it)
- cbz: .cbz (collection of images) files, preferably created with Fanfix (but any .cbz file is supported, though without most of Fanfix metadata, obviously)
- html: HTML files that you can open with any browser; note that it will create a directory structure with ```index.html``` as the main file -- we only support importing HTML files created by Fanfix

### Supported platforms

Any platform with at lest Java 1.6 on it should be ok.

It has been tested on Linux (Debian, Slackware, Ubuntu), MacOS X and Windows for now, but feel free to inform us if you try it on another system.

If you have any problems to compile it with a supported Java version (1.6+), please contact us.

## Options

You can start the program in two ways:

- ```java -jar fanfix.jar```
- ```fanfix``` (if you used *make install*)

The following arguments are allowed:

- ```--import [URL]```: import the story at URL into the local library
- ```--export [id] [output_type] [target]```: export the story denoted by ID to the target file
- ```--convert [URL] [output_type] [target] (+info)```: convert the story at URL into target, and force-add the .info and cover if +info is passed
- ```--read [id] ([chapter number])```: read the given story denoted by ID from the library
- ```--read-url [URL] ([chapter number])```: convert on the fly and read the story at URL, without saving it
- ```--search```: list the supported websites (```where```)
- ```--search [where] [keywords] (page [page]) (item [item])```: search on the supported website and display the given results page of stories it found, or the story details if asked
- ```--tag [where]```: list all the tags supported by this website
- ```--tag [index 1]... (page [page]) (item [item])```: search for the given stories or subtags, tag by tag, and display information about a specific page of results or about a specific item if requested
- ```--list```: list the stories present in the library and their associated IDs
- ```--server [key] [port]```: start a story server on this port
- ```--stop-server [key] [port]```: stop the remote server running on this port (key must be set to the same value)
- ```--remote [key] [host] [port]```: contact this server instead of the usual library (key must be set to the same value)
- ```--help```: display the available options
- ```--version```: return the version of the program

### Environment

Some environment variables are recognized by the program:

- ```LANG=en```: force the language to English
- ```CONFIG_DIR=$HOME/.fanfix```: use the given directory as a config directory (and copy the default configuration if needed)
- ```NOUTF=1```: try to fallback to non-unicode values when possible (can have an impact on the resulting files, not only on user messages)
- ```DEBUG=1```: force the ```DEBUG=true``` option of the configuration file (to show more information on errors)

## Compilation

```./configure.sh && make```

You can also import the java sources into, say, [Eclipse](https://eclipse.org/), and create a runnable JAR file from there.

There are some unit tests you can run, too:

```./configure.sh && make build test run-test```

If you run the unit tests, note that some flag files can impact them:

- ```test/VERBOSE```      : enable verbose mode
- ```test/OFFLINE```      : to forbid any downloading
- ```test/URLS```         : to allow testing URLs
- ```test/FORCE_REFRESH```: to force a clear of the cache

Note that ```test/CACHE``` can be kept, as it will contain all internet related files you need (if you allow URLs, run the test once which will populate the CACHE then go OFFLINE, it will still work).

The test files will be:

- ```test/*.url```  : URL to download in text format, content = URL
- ```test/*.story```: text mode story, content = story


### Dependant libraries (included)

Required:

- ```libs/nikiroo-utils-sources.jar```: some shared utility functions
- [```libs/unbescape-sources.jar```](https://github.com/unbescape/unbescape): a nice library to escape/unescape a lot of text formats; used here for HTML
- [```libs/jsoup-sources.jar```](https://jsoup.org/): a library to parse HTML
- [```libs/JSON-java-20190722-sources.jar```](https://github.com/stleary/JSON-java): a library to parse JSON

Optional:

- [```libs/jexer-sources.jar```](https://github.com/klamonte/jexer): a small library that offers TUI widgets
- [```pandoc```](http://pandoc.org/): to generate the man pages from the README files

Nothing else but Java 1.6+.

Note that calling ```make libs``` will export the libraries into the src/ directory.

## Author

Fanfix was written by Niki Roo <niki@nikiroo.be>
Commit	Line	Data
	1	English [Français](README-fr.md)
	2
	3	# Fanfix
	4
	5	Fanfix is a small Java program that can download stories from some supported websites and render them offline.
	6
	7	## 🔴 This is the command line and server program
	8
	9	You can also use:
	10	- the graphical client [Fanfix-swing](https://github.com/nikiroo/fanfix-swing/)
	11	- the TUI client [Fanfix-jexer](https://github.com/nikiroo/fanfix-jexer/)
	12
	13	## Synopsis
	14
	15	- ```fanfix``` --import [URL]
	16	- ```fanfix``` --export [id] [output_type] [target]
	17	- ```fanfix``` --convert [URL] [output_type] [target] (+info)
	18	- ```fanfix``` --read [id] ([chapter number])
	19	- ```fanfix``` --read-url [URL] ([chapter number])
	20	- ```fanfix``` --search
	21	- ```fanfix``` --search [where] [keywords] (page [page]) (item [item])
	22	- ```fanfix``` --search-tag
	23	- ```fanfix``` --search-tag [index 1]... (page [page]) (item [item])
	24	- ```fanfix``` --list
	25	- ```fanfix``` --server [key] [port]
	26	- ```fanfix``` --stop-server [key] [port]
	27	- ```fanfix``` --remote [key] [host] [port]
	28	- ```fanfix``` --help
	29
	30	## Description
	31
	32	(If you are interested in the recent changes, please check the [Changelog](changelog.md) -- note that starting from version 1.4.0, the changelog is checked at startup.)
	33
	34	This program will convert from a (supported) URL to an .epub file for stories or a .cbz file for comics (a few other output types are also available, like Plain Text, LaTeX, HTML...).
	35
	36	To help organize your stories, it can also work as a local library so you can:
	37
	38	- Import a story from its URL (or just from a file)
	39	- Export a story to a file (in any of the supported output types)
	40	- Display a story from the local library in text format in the console
	41	- Via [fanfix-swing](https://github.com/nikiroo/fanfix-swing/): Display a story from the local library graphically by itself ([fanfix-swing](https://github.com/nikiroo/fanfix-swing/)) or by calling a native program to handle it (potentially converted into HTML before hand, so any browser can open it)
	42
	43	### Supported websites
	44
	45	Currently, the following websites are supported:
	46
	47	- http://FimFiction.net/: fan fictions devoted to the My Little Pony show
	48	- http://Fanfiction.net/: fan fictions of many, many different universes, from TV shows to novels to games
	49	- http://mangahub.io/: a well filled repository of mangas (English)
	50	- https://e621.net/: a Furry website supporting comics, including MLP
	51	- https://sofurry.com/: same thing, but story-oriented
	52	- https://e-hentai.org/: done upon request (so, feel free to ask for more websites!)
	53	- http://mangas-lecture-en-ligne.fr/: a website offering a lot of mangas (in French)
	54
	55	### Support file types
	56
	57	We support a few file types for local story conversion (both as input and as output):
	58
	59	- epub: .epub files created by this program (we do not support "all" .epub files, at least for now)
	60	- text: local stories encoded in plain text format, with a few specific rules:
	61	- the title must be on the first line
	62	- the author (preceded by nothing, ```by ``` or ```©```) must be on the second line, possibly with the publication date in parenthesis (i.e., ```By Unknown (3rd October 1998)```)
	63	- chapters must be declared with ```Chapter x``` or ```Chapter x: NAME OF THE CHAPTER```, where ```x``` is the chapter number
	64	- a description of the story must be given as chapter number 0
	65	- a cover image may be present with the same filename as the story, but a .png, .jpeg or .jpg extension
	66	- info_text: contains the same information as the text format, but with a companion .info file to store some metadata (the .info file is supposed to be created by Fanfix or compatible with it)
	67	- cbz: .cbz (collection of images) files, preferably created with Fanfix (but any .cbz file is supported, though without most of Fanfix metadata, obviously)
	68	- html: HTML files that you can open with any browser; note that it will create a directory structure with ```index.html``` as the main file -- we only support importing HTML files created by Fanfix
	69
	70	### Supported platforms
	71
	72	Any platform with at lest Java 1.6 on it should be ok.
	73
	74	It has been tested on Linux (Debian, Slackware, Ubuntu), MacOS X and Windows for now, but feel free to inform us if you try it on another system.
	75
	76	If you have any problems to compile it with a supported Java version (1.6+), please contact us.
	77
	78	## Options
	79
	80	You can start the program in two ways:
	81
	82	- ```java -jar fanfix.jar```
	83	- ```fanfix``` (if you used make install)
	84
	85	The following arguments are allowed:
	86
	87	- ```--import [URL]```: import the story at URL into the local library
	88	- ```--export [id] [output_type] [target]```: export the story denoted by ID to the target file
	89	- ```--convert [URL] [output_type] [target] (+info)```: convert the story at URL into target, and force-add the .info and cover if +info is passed
	90	- ```--read [id] ([chapter number])```: read the given story denoted by ID from the library
	91	- ```--read-url [URL] ([chapter number])```: convert on the fly and read the story at URL, without saving it
	92	- ```--search```: list the supported websites (```where```)
	93	- ```--search [where] [keywords] (page [page]) (item [item])```: search on the supported website and display the given results page of stories it found, or the story details if asked
	94	- ```--tag [where]```: list all the tags supported by this website
	95	- ```--tag [index 1]... (page [page]) (item [item])```: search for the given stories or subtags, tag by tag, and display information about a specific page of results or about a specific item if requested
	96	- ```--list```: list the stories present in the library and their associated IDs
	97	- ```--server [key] [port]```: start a story server on this port
	98	- ```--stop-server [key] [port]```: stop the remote server running on this port (key must be set to the same value)
	99	- ```--remote [key] [host] [port]```: contact this server instead of the usual library (key must be set to the same value)
	100	- ```--help```: display the available options
	101	- ```--version```: return the version of the program
	102
	103	### Environment
	104
	105	Some environment variables are recognized by the program:
	106
	107	- ```LANG=en```: force the language to English
	108	- ```CONFIG_DIR=$HOME/.fanfix```: use the given directory as a config directory (and copy the default configuration if needed)
	109	- ```NOUTF=1```: try to fallback to non-unicode values when possible (can have an impact on the resulting files, not only on user messages)
	110	- ```DEBUG=1```: force the ```DEBUG=true``` option of the configuration file (to show more information on errors)
	111
	112	## Compilation
	113
	114	```./configure.sh && make```
	115
	116	You can also import the java sources into, say, [Eclipse](https://eclipse.org/), and create a runnable JAR file from there.
	117
	118	There are some unit tests you can run, too:
	119
	120	```./configure.sh && make build test run-test```
	121
	122	If you run the unit tests, note that some flag files can impact them:
	123
	124	- ```test/VERBOSE``` : enable verbose mode
	125	- ```test/OFFLINE``` : to forbid any downloading
	126	- ```test/URLS``` : to allow testing URLs
	127	- ```test/FORCE_REFRESH```: to force a clear of the cache
	128
	129	Note that ```test/CACHE``` can be kept, as it will contain all internet related files you need (if you allow URLs, run the test once which will populate the CACHE then go OFFLINE, it will still work).
	130
	131	The test files will be:
	132
	133	- ```test/*.url``` : URL to download in text format, content = URL
	134	- ```test/*.story```: text mode story, content = story
	135
	136
	137	### Dependant libraries (included)
	138
	139	Required:
	140
	141	- ```libs/nikiroo-utils-sources.jar```: some shared utility functions
	142	- [```libs/unbescape-sources.jar```](https://github.com/unbescape/unbescape): a nice library to escape/unescape a lot of text formats; used here for HTML
	143	- [```libs/jsoup-sources.jar```](https://jsoup.org/): a library to parse HTML
	144	- [```libs/JSON-java-20190722-sources.jar```](https://github.com/stleary/JSON-java): a library to parse JSON
	145
	146	Optional:
	147
	148	- [```libs/jexer-sources.jar```](https://github.com/klamonte/jexer): a small library that offers TUI widgets
	149	- [```pandoc```](http://pandoc.org/): to generate the man pages from the README files
	150
	151	Nothing else but Java 1.6+.
	152
	153	Note that calling ```make libs``` will export the libraries into the src/ directory.
	154
	155	## Author
	156
	157	Fanfix was written by Niki Roo <niki@nikiroo.be>
	158