Merge pull request #5 from schemen/devel

Merge Devel into master!

Author: schemen

.gitignore

@@ -7,4 +7,5 @@ comic/*
 kindlegen.exe
 .idea
 venv
-main.db
+main.db
+log/*

README.md

@@ -46,7 +46,29 @@ As a concept, M2em has different workers that run in a loop. All Chapter/user da
 * Converter - Converting images into ebooks
 * Sender - Compiling & Sending Emails to users and marking them as SENT
 
-With the sourceparser you can add support of a Webhost.
+
+### The Loop Run & Daemon Loop Run
+If you start m2em in loop mode (with or without --daemon) it will only consider any action with elements that that are
+younger than 24h hours.
+
+The use of that is having it running on the server 24/7, waiting for updates from the feeds and ONLY handling said updates.
+
+### Direct action
+You can start a part of the loop without the restriction of 24h. Use the -a (--action) command with either element you wish to start.
+
+Example: if you wish to download all chapters you have saved in your database, you start the download action.
+```
+./m2em.py --action downloader
+```
+### Chapter action
+You can directly apply an action to one chapter with the options --download, --convert or --send. You need to pass
+the ID of said chapter, you can find that out with "-Lc" or "-lc".
+You can pass multiple IDs.
+Example:
+```
+./m2em.py --download 100 #Downloads chapter with ID 100
+```
+
 
 ## Supported Websites
 * Mangastream
@@ -56,27 +78,42 @@ With the sourceparser you can add support of a Webhost.
 
 ### Help output:
 ```
-usage: m2em.py [-h] [-r RSS_FEED] [-u] [-l] [-L] [--list-feeds] [--list-users]
-               [-cd] [-s SWITCH_SEND] [-S SWITCH_CHAPTER] [-dc DELETE_CHAPTER]
-               [-du DELETE_USER] [-df DELETE_FEED] [--daemon] [-d]
+usage: m2em.py [-h] [-af ADD_FEED] [-au] [-lc] [-Lc] [-lf] [-lu] [-cd] [-s]
+               [--send [SEND [SEND ...]]] [--convert [CONVERT [CONVERT ...]]]
+               [--download [DOWNLOAD [DOWNLOAD ...]]] [-a ACTION]
+               [-ss SWITCH_SEND] [-sc SWITCH_CHAPTER] [-dc DELETE_CHAPTER]
+               [-du DELETE_USER] [-df DELETE_FEED] [--daemon] [-d] [-v]
 
 Manga to eManga - m2em
 
 optional arguments:
   -h, --help            show this help message and exit
-  -r RSS_FEED, --rss-feed RSS_FEED
+  -af ADD_FEED, --add-feed ADD_FEED
                         Add RSS Feed of Manga. Only Mangastream & MangaFox are
                         supported
-  -u, --add-user        Adds new user
-  -l, --list-chapters   Lists the last 10 Chapters
-  -L, --list-chapters-all
+  -au, --add-user       Adds new user
+  -lc, --list-chapters  Lists the last 10 Chapters
+  -Lc, --list-chapters-all
                         Lists all Chapters
-  --list-feeds          Lists all feeds
-  --list-users          Lists all Users
+  -lf, --list-feeds     Lists all feeds
+  -lu, --list-users     Lists all Users
   -cd, --create-db      Creates DB. Uses Configfile for Naming
-  -s SWITCH_SEND, --switch-send SWITCH_SEND
+  -s, --start           Starts one loop
+  --send [SEND [SEND ...]]
+                        Sends Chapter directly by chapter ID. Multiple IDs can
+                        be given
+  --convert [CONVERT [CONVERT ...]]
+                        Converts Chapter directly by chapter ID. Multiple IDs
+                        can be given
+  --download [DOWNLOAD [DOWNLOAD ...]]
+                        Downloads Chapter directly by chapter ID. Multiple IDs
+                        can be given
+  -a ACTION, --action ACTION
+                        Start action. Options are: rssparser (collecting feed
+                        data), downloader, converter or sender
+  -ss SWITCH_SEND, --switch-send SWITCH_SEND
                         Pass ID of User. Switches said user Send eBook status
-  -S SWITCH_CHAPTER, --switch-chapter SWITCH_CHAPTER
+  -sc SWITCH_CHAPTER, --switch-chapter SWITCH_CHAPTER
                         Pass ID of Chapter. Switches said Chapter Sent status
   -dc DELETE_CHAPTER, --delete-chapter DELETE_CHAPTER
                         Pass ID of Chapter. Deletes said Chapter
@@ -86,13 +123,15 @@ optional arguments:
                         Pass ID of Feed. Deletes said Feed
   --daemon              Run as daemon
   -d, --debug           Debug Mode
+  -v, --version         show program's version number and exit
+
 ```
 
 ## Initial Data
 To have a working environment you need to add some initial data and create the database
 ```x-sh
 ./m2em.py --create-db # Create a DB
-./m2em.py --rss-feed <URL> # Add an RSS Feed you want to pull
+./m2em.py --add-feed <URL> # Add an RSS Feed you want to pull
 # Please note that you should set the sending AFTER a complete run for now
 ./m2em.py --add-user  # Add a user
 
@@ -127,10 +166,19 @@ ServerStartSSL = True
 
 To start a single run through the workers, you can simply execute the main program:
 ```
-./m2em.py
+./m2em.py -s
+```
+
+If you wish to run the program as a daemon, start it with the option "--daemon" as well. It will re-run at the config "Sleep" in second.
+```
+./m2em.py -s --daemon
+```
+
+If you wish to disable/enable sending status of a user, use the -ss command
+```
+./m2em.py -ss <USERID>
 ```
 
-If you wish to run the program as a daemon, start it with the option "--daemon". It will re-run at the config "Sleep" in second.
 
 
 ### A complete run with nothing happening:
@@ -152,7 +200,6 @@ Finished sending ebooks!
 Everything else should be self-explanatory with the "-h" option.
 
 ## Known Issues
-* There is a huge data load in the beginning. It is recommended to only activate sending of Emails after one complete run
 * MangaFox has issues with SSL Verification on some systems. For now, Simply add the http feed.
 
 Please Open an issue if you find anything!

bin/_version.py

@@ -0,0 +1 @@
+__version__ = "v0.3.0"

bin/m2emConverter.py

@@ -5,70 +5,83 @@
 import bin.m2emHelper as helper
 
 
-def RecursiveConverter(config):
+class Converter:
 
-    # Load configs required here
-    database  = config["Database"]
-    saveloc   = config["SaveLocation"]
-    ebformat  = config["EbookFormat"]
-    ebprofile = config["EbookProfile"]
+    def __init__(self):
+        self.saveloc        = None
+        self.ebformat       = None
+        self.ebprofile      = None
+        self.mangatitle     = None
+        self.manganame      = None
+        self.imagefolder    = None
+        self.eblocation     = None
+        self.cbzlocation    = None
+        self.chapterdate    = None
 
-    # Load Chapters from Database
-    chapters = helper.getChapters(database)
 
 
-    # Start conversion loop!
-    for chapter in chapters:
+    def data_collector(self, config, chapter):
+
+        # Load configs required here
+        self.saveloc   = config["SaveLocation"]
+        self.ebformat  = config["EbookFormat"]
+        self.ebprofile = config["EbookProfile"]
+
 
         # get relevant data of this Manga
-        mangatitle   = chapter[2]
-        manganame    = chapter[11]
+        self.mangatitle   = chapter[2]
+        self.manganame    = chapter[11]
+        self.chapterdate  = chapter[3]
+
+        # check if mangatitle or manganame contains ":" characters that OS can't handle as folders
+        self.mangatitle = helper.sanetizeName(self.mangatitle)
+        self.manganame = helper.sanetizeName(self.manganame)
+
 
-        # check if mangatitle contains ":" characters that OS can't handle as folders
-        mangatitle = helper.sanetizeName(mangatitle)
+        # create folder variables
+        self.imagefolder  = str(self.saveloc + self.manganame + "/"+  self.mangatitle + "/images/")
+        self.eblocation   = str(self.saveloc + self.manganame + "/"+  self.mangatitle + "/" + self.mangatitle + "." + self.ebformat.lower())
+        self.cbzlocation  = str(self.saveloc + self.manganame + "/"+ self.mangatitle + "/" + self.mangatitle + ".cbz")
 
-        # check if manganame contains ":" characters that OS can't handle as folders
-        manganame = helper.sanetizeName(manganame)
 
-        imagefolder  = str(saveloc + manganame + "/"+  mangatitle + "/images/")
-        eblocation   = str(saveloc + manganame + "/"+  mangatitle + "/" + mangatitle + "." + ebformat.lower())
-        cbzlocation  = str(saveloc + manganame + "/"+ mangatitle + "/" + mangatitle + ".cbz")
 
 
+    def cbz_creator(self):
+
         # Create CBZ to make creation easier
-        if os.path.exists(cbzlocation):
-            logging.debug("Manga %s converted to CBZ already!" % mangatitle)
+        if os.path.exists(self.cbzlocation):
+            logging.debug("Manga %s converted to CBZ already!" % self.mangatitle)
         else:
-            logging.info("Starting conversion to CBZ of %s..." % mangatitle)
+            logging.info("Starting conversion to CBZ of %s..." % self.mangatitle)
 
 
             logging.debug("Opening CBZ archive...")
             try:
-                zf = zipfile.ZipFile(cbzlocation, "w")
+                zf = zipfile.ZipFile(self.cbzlocation, "w")
             except Exception as e:
                 logging.warning("Failed opening archive! %s" % e)
 
 
 
             logging.debug("Writing Images into CBZ")
-            for img in sorted(os.listdir(imagefolder)):
-                image = imagefolder + img
+            for img in sorted(os.listdir(self.imagefolder)):
+                image = self.imagefolder + img
                 logging.debug("Writing %s" % image)
                 zf.write(image,img)
 
             zf.close()
 
 
+
+    def eb_creator(self):
+
         # Start conversion to Ebook format!
-        if os.path.exists(eblocation):
-            logging.debug("Manga %s converted to Ebook already!" % mangatitle)
+        if os.path.exists(self.eblocation):
+            logging.debug("Manga %s converted to Ebook already!" % self.mangatitle)
         else:
-            logging.info("Starting conversion to Ebook of %s..." % mangatitle)
+            logging.info("Starting conversion to Ebook of %s..." % self.mangatitle)
 
             try:
-                subprocess.call(["kcc-c2e", "-p", ebprofile, "-f", ebformat, "-m", "-q", "-r",  "2", "-u", "-s",  cbzlocation])
+                subprocess.call(["kcc-c2e", "-p", self.ebprofile, "-f", self.ebformat, "-m", "-q", "-r",  "2", "-u", "-s",  self.cbzlocation])
             except Exception as e:
                 logging.debug("Failed to convert epub %s" % e)
-
-def ChapterConverter(imagelocation, config):
-    pass

bin/m2emConverterHandler.py

@@ -0,0 +1,86 @@
+import logging
+import os
+import bin.m2emHelper as helper
+from bin.m2emConverter import Converter
+
+
+def ConverterHandler(config, args):
+
+    # Load configs required here
+    database  = config["Database"]
+
+    # Load Chapters!
+    chapters = helper.getChapters(database)
+
+
+    # Start conversion loop!
+    for chapter in chapters:
+
+
+
+        # Verify if chapter has been downloaded already
+        if not helper.verifyDownload(config, chapter):
+            logging.debug("Manga %s has not been downloaded!" % chapter[2])
+        else:
+
+
+            # Spawn an Converter Object & get basic data from database & config
+            current_conversation = Converter()
+            current_conversation.data_collector(config,chapter)
+
+            # Check if Download loop & Download task is selected
+            if not args.start:
+                current_conversation.cbz_creator()
+                current_conversation.eb_creator()
+            else:
+
+                # Only start run if chapter is younger than 24h
+                if helper.checkTime(current_conversation.chapterdate):
+                    current_conversation.cbz_creator()
+                    current_conversation.eb_creator()
+                else:
+                    logging.debug("%s is older than 24h, will not be processed by daemon." % current_conversation.mangatitle)
+
+
+
+
+def directConverter(config, chapterids=[]):
+
+    logging.debug("Following Chapters are directly converted:")
+    logging.debug(chapterids)
+
+    # Load configs required here
+    database    = config["Database"]
+
+
+    chapters = helper.getChaptersFromID(database, chapterids)
+
+
+    if not chapters:
+        logging.error("No Chapters found with said ID!")
+    else:
+        # Start conversion loop!
+        for chapter in chapters:
+
+            # Verify if chapter has been downloaded already
+            if not helper.verifyDownload(config, chapter):
+                logging.info("Manga %s has not been downloaded!" % chapter[2])
+            else:
+
+
+                # Spawn an Converter Object & get basic data from database & config
+                current_conversation = Converter()
+                current_conversation.data_collector(config,chapter)
+
+                if os.path.exists(current_conversation.cbzlocation):
+                    logging.info("Manga %s converted to CBZ already!" % current_conversation.mangatitle)
+                else:
+                    logging.info("Starting conversion to CBZ of %s..." % current_conversation.mangatitle)
+                    current_conversation.cbz_creator()
+
+                # Start conversion to Ebook format!
+                if os.path.exists(current_conversation.eblocation):
+                    logging.info("Manga %s converted to Ebook already!" % current_conversation.mangatitle)
+                else:
+                    logging.info("Starting conversion to Ebook of %s..." % current_conversation.mangatitle)
+                    current_conversation.eb_creator()