-Jexer - Java Text User Interface library
-========================================
+English [Français](README-fr.md)
-This library implements a text-based windowing system loosely
-reminiscient of Borland's [Turbo
-Vision](http://en.wikipedia.org/wiki/Turbo_Vision) system. (For those
-wishing to use the actual C++ Turbo Vision library, see [Sergio
-Sigala's C++ version based on the sources released by
-Borland,](http://tvision.sourceforge.net/) or consider Free Pascal's
-[Free Vision library.](http://wiki.freepascal.org/Free_Vision))
+# Fanfix
-Jexer currently supports three backends:
+Fanfix is a small Java program that can download stories from some supported websites and render them offline.
-* System.in/out to a command-line ECMA-48 / ANSI X3.64 type terminal
- (tested on Linux + xterm). I/O is handled through terminal escape
- sequences generated by the library itself: ncurses is not required
- or linked to. xterm mouse tracking is supported using both UTF8 and
- SGR coordinates. Images are optionally rendered via sixel graphics
- (see jexer.ECMA48.sixel). For the demo application, this is the
- default backend on non-Windows/non-Mac platforms.
+## 🔴 This is the command line and server program
-* The same command-line ECMA-48 / ANSI X3.64 type terminal as above,
- but to any general InputStream/OutputStream or Reader/Writer. See
- the file jexer.demos.Demo2 for an example of running the demo over a
- TCP (telnet) socket. jexer.demos.Demo3 demonstrates how one might
- use a character encoding than the default UTF-8.
+You can also use:
+- the graphical client [Fanfix-swing](https://github.com/nikiroo/fanfix-swing/)
+- the TUI client [Fanfix-jexer](https://github.com/nikiroo/fanfix-jexer/)
-* Java Swing UI. The default window size for Swing is 80x25 and 20
- point font; this can be changed in the TApplication(BackendType)
- constructor. For the demo applications, this is the default backend
- on Windows and Mac platforms. This backend can be explicitly
- selected for the demo applications by setting jexer.Swing=true.
+## Synopsis
-Additional backends can be created by subclassing
-jexer.backend.Backend and passing it into the TApplication
-constructor. See Demo5 and Demo6 for examples of other backends.
+- ```fanfix``` --import [*URL*]
+- ```fanfix``` --export [*id*] [*output_type*] [*target*]
+- ```fanfix``` --convert [*URL*] [*output_type*] [*target*] (+info)
+- ```fanfix``` --read [*id*] ([*chapter number*])
+- ```fanfix``` --read-url [*URL*] ([*chapter number*])
+- ```fanfix``` --search
+- ```fanfix``` --search [*where*] [*keywords*] (page [*page*]) (item [*item*])
+- ```fanfix``` --search-tag
+- ```fanfix``` --search-tag [*index 1*]... (page [*page*]) (item [*item*])
+- ```fanfix``` --list
+- ```fanfix``` --server [*key*] [*port*]
+- ```fanfix``` --stop-server [*key*] [*port*]
+- ```fanfix``` --remote [*key*] [*host*] [*port*]
+- ```fanfix``` --help
-The Jexer homepage, which includes additional information and binary
-release downloads, is at: https://jexer.sourceforge.io . The Jexer
-source code is hosted at: https://gitlab.com/klamonte/jexer .
+## Description
+(If you are interested in the recent changes, please check the [Changelog](changelog.md) -- note that starting from version 1.4.0, the changelog is checked at startup.)
+This program will convert from a (supported) URL to an .epub file for stories or a .cbz file for comics (a few other output types are also available, like Plain Text, LaTeX, HTML...).
-License
--------
+To help organize your stories, it can also work as a local library so you can:
-This project is licensed under the MIT License. See the file LICENSE
-for the full license text.
+- Import a story from its URL (or just from a file)
+- Export a story to a file (in any of the supported output types)
+- Display a story from the local library in text format in the console
+- Via [fanfix-swing](https://github.com/nikiroo/fanfix-swing/): Display a story from the local library graphically **by itself** ([fanfix-swing](https://github.com/nikiroo/fanfix-swing/)) or **by calling a native program to handle it** (potentially converted into HTML before hand, so any browser can open it)
+### Supported websites
-Maven
------
+Currently, the following websites are supported:
-Jexer is available on Maven Central:
+- http://FimFiction.net/: fan fictions devoted to the My Little Pony show
+- http://Fanfiction.net/: fan fictions of many, many different universes, from TV shows to novels to games
+- http://mangahub.io/: a well filled repository of mangas (English)
+- https://e621.net/: a Furry website supporting comics, including MLP
+- https://sofurry.com/: same thing, but story-oriented
+- https://e-hentai.org/: done upon request (so, feel free to ask for more websites!)
+- http://mangas-lecture-en-ligne.fr/: a website offering a lot of mangas (in French)
-```xml
-<dependency>
- <groupId>com.gitlab.klamonte</groupId>
- <artifactId>jexer</artifactId>
- <version>0.3.0</version>
-</dependency>
-```
+### Support file types
+We support a few file types for local story conversion (both as input and as output):
+- epub: .epub files created by this program (we do not support "all" .epub files, at least for now)
+- text: local stories encoded in plain text format, with a few specific rules:
+ - the title must be on the first line
+ - the author (preceded by nothing, ```by ``` or ```©```) must be on the second line, possibly with the publication date in parenthesis (i.e., ```By Unknown (3rd October 1998)```)
+ - chapters must be declared with ```Chapter x``` or ```Chapter x: NAME OF THE CHAPTER```, where ```x``` is the chapter number
+ - a description of the story must be given as chapter number 0
+ - a cover image may be present with the same filename as the story, but a .png, .jpeg or .jpg extension
+- info_text: contains the same information as the text format, but with a companion .info file to store some metadata (the .info file is supposed to be created by Fanfix or compatible with it)
+- cbz: .cbz (collection of images) files, preferably created with Fanfix (but any .cbz file is supported, though without most of Fanfix metadata, obviously)
+- html: HTML files that you can open with any browser; note that it will create a directory structure with ```index.html``` as the main file -- we only support importing HTML files created by Fanfix
-Acknowledgements
-----------------
+### Supported platforms
-Jexer makes use of the Terminus TrueType font [made available
-here](http://files.ax86.net/terminus-ttf/) .
+Any platform with at lest Java 1.6 on it should be ok.
+It has been tested on Linux (Debian, Slackware, Ubuntu), MacOS X and Windows for now, but feel free to inform us if you try it on another system.
+If you have any problems to compile it with a supported Java version (1.6+), please contact us.
-Usage
------
+## Options
-Simply subclass TApplication and then run it in a new thread:
+You can start the program in two ways:
-```Java
-import jexer.*;
+- ```java -jar fanfix.jar```
+- ```fanfix``` (if you used *make install*)
-class MyApplication extends TApplication {
+The following arguments are allowed:
- public MyApplication() throws Exception {
- super(BackendType.SWING); // Could also use BackendType.XTERM
+- ```--import [URL]```: import the story at URL into the local library
+- ```--export [id] [output_type] [target]```: export the story denoted by ID to the target file
+- ```--convert [URL] [output_type] [target] (+info)```: convert the story at URL into target, and force-add the .info and cover if +info is passed
+- ```--read [id] ([chapter number])```: read the given story denoted by ID from the library
+- ```--read-url [URL] ([chapter number])```: convert on the fly and read the story at URL, without saving it
+- ```--search```: list the supported websites (```where```)
+- ```--search [where] [keywords] (page [page]) (item [item])```: search on the supported website and display the given results page of stories it found, or the story details if asked
+- ```--tag [where]```: list all the tags supported by this website
+- ```--tag [index 1]... (page [page]) (item [item])```: search for the given stories or subtags, tag by tag, and display information about a specific page of results or about a specific item if requested
+- ```--list```: list the stories present in the library and their associated IDs
+- ```--server [key] [port]```: start a story server on this port
+- ```--stop-server [key] [port]```: stop the remote server running on this port (key must be set to the same value)
+- ```--remote [key] [host] [port]```: contact this server instead of the usual library (key must be set to the same value)
+- ```--help```: display the available options
+- ```--version```: return the version of the program
- // Create standard menus for File and Window
- addFileMenu();
- addWindowMenu();
+### Environment
- // Add a custom window, see below for its code. The TWindow
- // constructor will add it to this application.
- new MyWindow(this);
- }
+Some environment variables are recognized by the program:
- public static void main(String [] args) throws Exception {
- MyApplication app = new MyApplication();
- (new Thread(app)).start();
- }
-}
-```
+- ```LANG=en```: force the language to English
+- ```CONFIG_DIR=$HOME/.fanfix```: use the given directory as a config directory (and copy the default configuration if needed)
+- ```NOUTF=1```: try to fallback to non-unicode values when possible (can have an impact on the resulting files, not only on user messages)
+- ```DEBUG=1```: force the ```DEBUG=true``` option of the configuration file (to show more information on errors)
-Similarly, subclass TWindow and add some widgets:
+## Compilation
-```Java
-class MyWindow extends TWindow {
+```./configure.sh && make```
- public MyWindow(TApplication application) {
- // See TWindow's API for several constructors. This one uses the
- // application, title, width, and height. Note that the window width
- // and height include the borders. The widgets inside the window
- // will see (0, 0) as the top-left corner inside the borders,
- // i.e. what the window would see as (1, 1).
- super(application, "My Window", 30, 20);
+You can also import the java sources into, say, [Eclipse](https://eclipse.org/), and create a runnable JAR file from there.
- // See TWidget's API for convenience methods to add various kinds of
- // widgets. Note that ANY widget can be a container for other
- // widgets: TRadioGroup for example has TRadioButtons as child
- // widgets.
+There are some unit tests you can run, too:
- // We will add a basic label, text entry field, and button.
- addLabel("This is a label", 5, 3);
- addField(5, 5, 20, false, "enter text here");
- // For the button, we will pop up a message box if the user presses
- // it.
- addButton("Press &Me!", 5, 8, new TAction() {
- public void DO() {
- MyWindow.this.messageBox("Box Title", "You pressed me, yay!");
- }
- } );
- }
-}
-```
+```./configure.sh && make build test run-test```
-Put these into a file, compile it with jexer.jar in the classpath, run
-it and you'll see an application like this:
+If you run the unit tests, note that some flag files can impact them:
-
+- ```test/VERBOSE``` : enable verbose mode
+- ```test/OFFLINE``` : to forbid any downloading
+- ```test/URLS``` : to allow testing URLs
+- ```test/FORCE_REFRESH```: to force a clear of the cache
+Note that ```test/CACHE``` can be kept, as it will contain all internet related files you need (if you allow URLs, run the test once which will populate the CACHE then go OFFLINE, it will still work).
+The test files will be:
-More Examples
--------------
+- ```test/*.url``` : URL to download in text format, content = URL
+- ```test/*.story```: text mode story, content = story
-The examples/ folder currently contains:
- * A [prototype tiling window
- manager](/examples/JexerTilingWindowManager.java) in less than 250
- lines of code.
+### Dependant libraries (included)
- * A [prototype image thumbnail
- viewer](/examples/JexerImageViewer.java) in less than 350 lines of
- code.
+Required:
-jexer.demos contains official demos showing all of the existing UI
-controls. The demos can be run as follows:
+- ```libs/nikiroo-utils-sources.jar```: some shared utility functions
+- [```libs/unbescape-sources.jar```](https://github.com/unbescape/unbescape): a nice library to escape/unescape a lot of text formats; used here for HTML
+- [```libs/jsoup-sources.jar```](https://jsoup.org/): a library to parse HTML
+- [```libs/JSON-java-20190722-sources.jar```](https://github.com/stleary/JSON-java): a library to parse JSON
- * 'java -jar jexer.jar' . This will use System.in/out with
- xterm-like sequences on non-Windows non-Mac platforms. On Windows
- and Mac it will use a Swing JFrame.
+Optional:
- * 'java -Djexer.Swing=true -jar jexer.jar' . This will always use
- Swing on any platform.
+- [```libs/jexer-sources.jar```](https://github.com/klamonte/jexer): a small library that offers TUI widgets
+- [```pandoc```](http://pandoc.org/): to generate the man pages from the README files
- * 'java -cp jexer.jar jexer.demos.Demo2 PORT' (where PORT is a
- number to run the TCP daemon on). This will use the telnet
- protocol to establish an 8-bit clean channel and be aware of
- screen size changes.
+Nothing else but Java 1.6+.
- * 'java -cp jexer.jar jexer.demos.Demo3' . This will use
- System.in/out with xterm-like sequences. One can see in the code
- how to pass a different InputReader and OutputReader to
- TApplication, permitting a different encoding than UTF-8.
+Note that calling ```make libs``` will export the libraries into the src/ directory.
- * 'java -cp jexer.jar jexer.demos.Demo4' . This demonstrates hidden
- windows and a custom TDesktop.
+## Author
- * 'java -cp jexer.jar jexer.demos.Demo5' . This demonstrates two
- demo applications using different fonts in the same Swing frame.
+Fanfix was written by Niki Roo <niki@nikiroo.be>
- * 'java -cp jexer.jar jexer.demos.Demo6' . This demonstrates two
- applications performing I/O across three screens: an xterm screen
- and Swing screen, monitored from a third Swing screen.
-
-
-
-More Screenshots
-----------------
-
-
-
-
-
-
-
-
-
-
-Terminal Support
-----------------
-
-The table below lists terminals tested against Jexer's ECMA48/Xterm
-backend.
-
-| Terminal | Environment | Mouse Click | Mouse Cursor | Images |
-| -------------- | ------------------ | ----------- | ------------ | ------ |
-| xterm | X11 | yes | yes | yes |
-| lcxterm(3) | CLI, Linux console | yes | yes | no |
-| rxvt-unicode | X11 | yes | yes | no |
-| alacritty(3) | X11 | yes | yes | no |
-| gnome-terminal | X11 | yes | yes | no |
-| xfce4-terminal | X11 | yes | yes | no |
-| mlterm | X11 | yes | yes | no(5) |
-| aminal(3) | X11 | yes | no | no |
-| konsole | X11 | yes | no | no |
-| yakuake | X11 | yes | no | no |
-| screen | CLI | yes(1) | yes(1) | no(2) |
-| tmux | CLI | yes(1) | yes(1) | no |
-| putty | X11, Windows | yes | no | no(2) |
-| Linux | Linux console | no | no | no(2) |
-| qodem(3) | CLI, Linux console | yes | yes(4) | no |
-| qodem-x11(3) | X11 | yes | no | no |
-
-1 - Requires mouse support from host terminal.
-
-2 - Also fails to filter out sixel data, leaving garbage on screen.
-
-3 - Latest in repository.
-
-4 - Requires TERM=xterm-1003 before starting.
-
-5 - Opening image crashes terminal.
-
-
-
-System Properties
------------------
-
-The following properties control features of Jexer:
-
- jexer.Swing
- -----------
-
- Used only by jexer.demos.Demo1 and jexer.demos.Demo4. If true, use
- the Swing interface for the demo application. Default: true on
- Windows (os.name starts with "Windows") and Mac (os.name starts with
- "Mac"), false on non-Windows and non-Mac platforms.
-
- jexer.Swing.cursorStyle
- -----------------------
-
- Used by jexer.backend.SwingTerminal. Selects the cursor style to
- draw. Valid values are: underline, block, outline. Default:
- underline.
-
- jexer.Swing.tripleBuffer
- ------------------------
-
- Used by jexer.backend.SwingTerminal. If true, use triple-buffering
- which reduces screen tearing but may also be slower to draw on
- slower systems. If false, use naive Swing thread drawing, which may
- be faster on slower systems but also more likely to have screen
- tearing. Default: true.
-
- jexer.TTerminal.ptypipe
- -----------------------
-
- Used by jexer.TTerminalWindow. If true, spawn shell using the
- 'ptypipe' utility rather than 'script'. This permits terminals to
- resize with the window. ptypipe is a separate C language utility,
- available at https://gitlab.com/klamonte/ptypipe. Default: false.
-
- jexer.TTerminal.closeOnExit
- ---------------------------
-
- Used by jexer.TTerminalWindow. If true, close the window when the
- spawned shell exits. Default: false.
-
- jexer.ECMA48.rgbColor
- ---------------------
-
- Used by jexer.backend.ECMA48Terminal. If true, emit T.416-style RGB
- colors for normal system colors. This is expensive in bandwidth,
- and potentially terrible looking for non-xterms. Default: false.
-
- jexer.ECMA48.sixel
- ------------------
-
- Used by jexer.backend.ECMA48Terminal. If true, emit image data
- using sixel, otherwise show blank cells where images could be. This
- is expensive in bandwidth, very expensive in CPU (especially for
- large images), and will leave artifacts on the screen if the
- terminal does not support sixel. Default: true.
-
-
-
-Known Issues / Arbitrary Decisions
-----------------------------------
-
-Some arbitrary design decisions had to be made when either the
-obviously expected behavior did not happen or when a specification was
-ambiguous. This section describes such issues.
-
- - See jexer.tterminal.ECMA48 for more specifics of terminal
- emulation limitations.
-
- - TTerminalWindow uses cmd.exe on Windows. Output will not be seen
- until enter is pressed, due to cmd.exe's use of line-oriented
- input (see the ENABLE_LINE_INPUT flag for GetConsoleMode() and
- SetConsoleMode()).
-
- - TTerminalWindow by default launches 'script -fqe /dev/null' or
- 'script -q -F /dev/null' on non-Windows platforms. This is a
- workaround for the C library behavior of checking for a tty:
- script launches $SHELL in a pseudo-tty. This works on Linux and
- Mac but might not on other Posix-y platforms.
-
- - Closing a TTerminalWindow without exiting the process inside it
- may result in a zombie 'script' process.
-
- - When using the Swing backend, and not using 'ptypipe', closing a
- TTerminalWindow without exiting the process inside it may result
- in a SIGTERM to the JVM causing it to crash. The root cause is
- currently unknown, but is potentially a bug in more recent
- releases of the 'script' utility from the util-linux package.
-
- - TTerminalWindow can only notify the child process of changes in
- window size if using the 'ptypipe' utility, due to Java's lack of
- support for forkpty() and similar. ptypipe is available at
- https://gitlab.com/klamonte/ptypipe.
-
- - Java's InputStreamReader as used by the ECMA48 backend requires a
- valid UTF-8 stream. The default X10 encoding for mouse
- coordinates outside (160,94) can corrupt that stream, at best
- putting garbage keyboard events in the input queue but at worst
- causing the backend reader thread to throw an Exception and exit
- and make the entire UI unusable. Mouse support therefore requires
- a terminal that can deliver either UTF-8 coordinates (1005 mode)
- or SGR coordinates (1006 mode). Most modern terminals can do
- this.
-
- - jexer.session.TTYSession calls 'stty size' once every second to
- check the current window size, performing the same function as
- ioctl(TIOCGWINSZ) but without requiring a native library.
-
- - jexer.backend.ECMA48Terminal calls 'stty' to perform the
- equivalent of cfmakeraw() when using System.in/out. System.out is
- also (blindly!) put in 'stty sane cooked' mode when exiting.
-
- - jexer.backend.ECMA48Terminal uses a single palette containing
- MAX_COLOR_REGISTERS colors for all sixel images. These colors are
- generated in the SixelPalette.makePalette() method with bits for
- hue, saturation, and luminance, and the two extremes set to pure
- black and pure white. This provides a reasonable general-purpose
- palette light on CPU, but at a cost that individual images do not
- look as good as the terminal is actually capable of.
-
-
-
-See Also
---------
-
-[Tranquil Java IDE](https://tjide.sourceforge.io) is a TUI-based
-integrated development environment for the Java language that was
-built using a very lightly modified GPL version of Jexer. TJ provided
-a real-world use case to shake out numerous bugs and limitations of
-Jexer.
-
-
-
-Maintainers Wanted
-------------------
-
-Both Jexer and TJIDE are seeking additional maintainers. I am not in
-a position in life to take on significant off-hours programming work,
-and am willing to hand these projects over to one or more persons with
-time and interest.
-
-My personal code design philosophy for TJIDE/Jexer is outlined at
-https://gitlab.com/klamonte/tjide/blob/master/java/docs/code_design.txt
-. I realize that some of the features listed below may require
-deviations from this philosophy, but this is what I have built so far.
-
-Some of the areas that will likely require significant efforts are:
-
- * Editor improvements. The editor is currently very minimalistic,
- much closer to MS-DOS edit.com than a real programmer's editor.
- Users will probably desire many more features: drag-and-drop, real
- syntax or at least regexp highlighting (not just keywords), paren
- matching, paragraph/comment reflow, and dozens more. The
- underlying Document/Line/Word model is not going to be sufficient
- to meet these features.
-
- * Better Windows and OSX support. It would be nice to ship a
- jlink'ed JVM on these platforms with the JRE, JDK, and JPDA
- modules all together. For Windows, it might be preferable to
- consider doing any of the following: ship a third-party terminal,
- use PowerShell, or use the newer ConPTY for TTerminalWindow.
-
- * Bug fixes. The Jexer codebase is quite large despite my best
- efforts. Bugs are typically very small to fix, but can take some
- time to find: a simple NPE or AssertionError can sometimes take
- 4-8 hours to squash. Fortunately, fixing issues in one place has
- not often led to breakages elsewhere.
-
- * New Jexer applications. So far as I know, Jexer is the only
- mouse-supporting full TUI windowing framework with sixel image
- support in existence. I cannot predict what kinds of applications
- could be built out of it, and how those needs will push back to
- the framework.
-
-These are what I can clearly see right now. Obviously users are
-capable of finding many more.
-
-I intend to continue poking on Jexer and TJIDE, and will maintain a
-branch to be "the fastest and simplest Java language IDE available",
-which will deliberately remain small.
-
-I hope that other languages choose to transliterate Jexer to provide
-TUIs to their own platforms. I will be happy to help them understand
-the code to support those efforts.
git://git.nikiroo.be / nikiroo-utils.git / blobdiff