Quantcast
Channel: MobileRead Forums - Calibre
Viewing all articles
Browse latest Browse all 31520

How I Manage eBooks with calibre

$
0
0
Revised 2012-07-20


Strategy
Spoiler:


Experience:
  • Gain Experience. If new to calibre, first spend a few months becoming familiar with it. See the suggestions in the "Help" section. Some experience with calibre and ebooks will provide a foundation for identifying personal needs and developing personal strategies for ebook management.
  • Example, Experience. I started collecting ebooks and using calibre to manage them in December 2010 when I bought a Kindle. Learning about ebooks and calibre kept me busy for months. I did not begin to clarify my needs or define personal strategies for ebook management until after about eight months.

Needs:
  • Identify Needs. Identify current personal needs regarding ebooks.
  • Examples, My Current Needs:
  • Feed the maw of a voracious reading habit.
  • Read usually at home rather than elsewhere, sometimes while out and about.
  • Read mostly text-based speculative fiction and paranormal romance, some popular non-fiction and news.
  • Occasionally read books with complex formats, graphics, or technical content.
  • Avoid reading books with annoying format problems.
  • Avoid wasting time with unnecessary work to fix format problems.

Strategies:
  • Develop Strategies. Develop personal strategies for ebook management. Derive strategies from needs within the constraints of current technology and resources.
  • Examples, My Current Strategies:
  • Use calibre as the hub of my ebook universe.
  • Use other tools and devices as temporary peripherals, for fixing or reading books.
  • Keep it simple.
  • Be discriminating about book source and quality of formatting. Some sources tend to have books with higher quality formatting than other sources. Higher quality formatting means fewer format problems per book. Needing to fix fewer format problems results in less annoyance at having to fix someone else's work, and more time for reading. The higher the quality of formatting, the more I enjoy reading.
  • Be discriminating about content. That means being selective about which authors to read, an author's overall writing skill, and which subject matter or genre falls within my areas of interest. The better the writing in a book, and the higher my interest in its content, the more I enjoy reading.
  • Get and process books one author at a time. For authors with a lot of books, do that author in stages, one series at a time. This is a simple and small scope for limiting the number of books needing processing, which circumvents feeling overwhelmed facing the prospect of processing a lot of books when I'd rather just read.
  • Standardize on EPUB format for the master copy of most books. Make exceptions for complexly formatted books such as technical or text books in PDF format, which are usually best kept as PDF masters for reading in PDF format. Better tools are available for fixing EPUB than for fixing MOBI or other MOBI-related Amazon formats.
  • Derive Workflow. Derive workflow from needs and strategy, within the context of existing technological resources and skill using them.
  • See Other Workflow Discussion. For additional discussion and examples of calibre-related strategy and workflow, see post by kiwidude and followup post by kiwidude.
  • Examples, My Workflow. The workflow I use for processing books is laid out after the "Preferences" and "Customization" sections. Each workflow section is itself part of an overall book processing sequence that flows from "Getting Books" to "Reading Books". In real life I vary details of the workflow when convenient—it is not set in stone. Specific workflow steps often depend on specific calibre preference settings, or customizations such as a custom column or third-party plugin.
  • Seek Congruence. Gradually evolve needs, strategy, workflow, skills, preferences, and customizations toward mutual congruence.


Preferences
Spoiler:

Preferences, Overall:
  • Changing Preference Settings. After initially installing calibre and using the welcome wizard to set language, library path, device, and email information, other calibre preference settings will remain at default until they are intentionally changed. Eventually changing specific preference settings may suit personal needs, strategies, or workflow better. Test a change enough to know what it does before using it more permanently.
  • Example Preference Settings. My current preference settings are listed below. Some settings are default. Some change over time as I learn more, calibre changes, or needs, strategy, or workflow change. So these settings are examples, not recommendations.

Example Preferences, Interface:
Spoiler:
  • Look and Feel, Main Interface. User interface style: System default. User interface layout: Narrow—puts the Book Details display below the booklist rather than to the right, and allows more horizontal space to see columns in the booklist. Choose language: English. Checked, Show tooltips in the book list. Checked, Show splash screen at startup. Unchecked, all others. Interface font: Lucida Grande 13. Icon size: small. Show text under icons: never—after the first few months with text under the icons, I found I didn't need the text anymore.
  • Look and Feel, Book Details. Displayed Metadata, Checked: Title, Series, Authors, Identifiers, Path, Comments, and custom columns FQ (Format Quality), Genres, Kinds, and Status. Unchecked, Use Roman numerals for series (box located at top right)—Roman numerals are not easy to understand at a glance. The Default author link template is useful for getting more author and title information by double-clicking a name under Authors in the Book Details panel. The default author link template for wikipedia is good for popular authors:
    Code:

    http://en.wikipedia.org/w/index.php?search={author}
    I replaced that with this author link template for Fantastic Fiction (copied from the third-party plugin Search the Internet):
    Code:

    http://www.fantasticfiction.co.uk/search/?searchfor=author&keywords={author}
  • Look and Feel, Tag Browser. Tag browser category partitioning method, disabled—I prefer a flat rather than hierarchical tag structure. Unchecked, Show average ratings—I don't use the default Rating column or downloaded ratings. Unchecked, Use alternating row colors in Tag Browser.
  • Look and Feel, Cover Browser. Unchecked, Show in separate window. Unchecked, When Showing use fullscreen. I don't use Cover Browser.
  • Look and Feel, Column Coloring. I use a "!q0" value in the custom column #fq (format quality) to set red text in the following columns of a "!q0" record: Authors, Series, Title, timestamp (Date), and FQ (format quality). Wishlist/placeholder with red text in those fields are easily distinguished from real books in the booklist.
  • Behavior. Checked, Show notification of new version. Checked, Yes/No columns have 3 values—if I used yes/no columns I'd probably want the Yes, No, and Blank option, rather than just Yes and No where No includes any not yet assigned in addition to those specifically assigned No. Unchecked, both News options—I like to manually control when Newsfeeds are sent to device or deleted, and also I don't read News often. Preferred output format: EPUB—useful when adding books because EPUB displays fastest in calibre viewer and is easier to edit when necessary, although I mostly read MOBI on Kindle. Default network timeout: 5 seconds. Job priority: Normal. Restriction to apply when opening library: Blank (none)—I usually feel restricted in restrictions when I forget I'm in one and try to search for something outside the restriction. Edit metadata (single) layout: Default. Preferred input format order: 1 EPUB, 2 MOBI, 3 LIT, others default order. Use internal viewer for: checked, all formats—this is helpful after Adding books for initial assessment of format using the calibre Viewer, and it is easy enough to use the third-party plugin Open With if necessary for editing or accurately rendering a format in another application.
  • Add Your Own Columns. Columns are discussed below in the "Customization" section.
  • Toolbar. Main Toolbar and Context Menu for Library are set to my convenience, and Main Toolbar with Device Connected is set to match Main Toolbar as closely as possible primarily so the contrast between Toolbars won't be startling when connecting or ejecting devices. Context Menu for Library is where I install plugin commands that work on current book selection. I use the Main Toolbar for the primary calibre icons, any plugin icons that apply commands for all books in the library, plus plugins that are frequently used even if they apply commands only to selected books. I set the Menubar with only Choose Library, Preferences, and Help. I set Menubar with Device Connected with only Preferences and Help. I don't use Optional Second Toolbar, Context Menu for Device, or Context Menu for Cover Browser.
  • Searching. Unchecked, Search as you type—allows faster performance. Unchecked, Highlight search results instead of restricting the booklist to the results—unchecked this groups search results all together in their own list, while checked it moves the results to wherever they belong in the sort order of the larger booklist while highlighting only the books of the search result; the Highlighting button (3 blue lines with red slash to the right of the search box above booklist) toggles this behavior on or off. Unchecked, Limit the searched metadata for searching without a column prefix. Note that in large libraries prefixed searches such as "author:heinlein" rather than non-prefixed searches such as "heinlein" are much faster, so it is worthwhile to get in the habit of using prefixed searches. Grouped Search Terms, not used.


Example Preferences, Conversion:
Spoiler:
  • Conversion, Input Options, Comic Input. Checked, Disable conversion of images to black and white—I prefer reading comics in color on iPad rather than in black and white on e-ink Kindle.
  • Conversion, Common Options, Page Setup, Output Profile. Set to Kindle—I leave this intentionally set on Kindle, because I read most books on Kindle and most of my books are text-based fiction with few graphics. I use the iPad less frequently for reading books with complex layouts, graphics-laden content like comics, or technical material with source code or equations. If necessary, doing another conversion with choice of iPad as Output Profile in the Conversion Dialog will temporarily override the Kindle setting during the conversion.
  • Conversion, Common Options, Page Setup, Input Profile. Set to Default Input Profile.
  • Conversion, Output Options, MOBI Output. Checked, Use author sort for author—sets author sort into author field in the format, so the Kindle will sort author correctly although it displays First_Name Last_Name. Unchecked, all other Mobi Output Options.
  • Conversion Miscellany. All other conversion options are default.


Example Preferences, Import/Export:
Spoiler:
  • Adding Books, The Add Process, Reading Metadata. Checking "Read metadata from file contents rather than filename" sometimes works okay for retail books in EPUB or MOBI formats, but can provide unpredictable and non-standard results from the metadata internal to the format. So I don't use it. Unchecked, "Read metadata from file contents rather than filename" uses the Regular Expression in the Regular Expression menu box to guess metadata from the filename—this works well for books from poor quality sources or for formats that don't include the metadata internally, but works best if the incoming books' filenames are first modified out in the operating system to match the Regular Expression. I use the Unchecked setting, along with this regular expression:
    Code:

    (?P<author>[^_]+?) - ((?P<series>.*) (?P<series_index>[0-9]*) - )?(?P<title>.+)
  • Adding Books, The Add Process, Other Options. Unchecked, Swap author last and first name—this is easier to do with the third-party plugin Quality Check or with Edit Metadata in Bulk's Search and Replace after the Add than switching it back and forth in Preferences. Checked, When using 'Copy To Library' preserve the date—it seems more useful to know when the book first arrived in any of my libraries, rather than when it was transferred to the Current Library. Unchecked, Automerge—I want to assess any duplicates or variations of multiple formats for a book before keeping the best of the formats, which is easiest to do when incoming formats for the same book are not merged into the same book record. Tags to apply when adding a book: "!new"—meaning "newly added", with the "!" making it unique for non-prefixed searches, and also sorting it to the front of the Tag column in booklist display, to be easily seen among multiple tags when the tag column is narrow to save horizontal display space.
  • Adding Books, Automatic Adding. I haven't needed to use this feature. Usually I add books by dragging and dropping directly into the calibre booklist window, only a few books at a time. I'd choose these options if I used it: Unchecked, Check for duplicates when auto-adding files—I'd rather use the third-party plugin Find Duplicates and compare duplicates later. Unchecked, Automatically convert added files to current output format—for large bulk adds, I'd want to be selective about what titles and formats I choose to convert. All unchecked, Ignore files with certain extensions—that would be the baseline, although selecting all but a few formats to ignore could be useful if tailored for any specific large bulk Add.
  • Saving Books to Disk. Checked, Save cover separately. Checked, Update metadata in saved copies. Checked, Save metadata in OPF file. Checked, Convert Non-English. Checked, Show files in file browser after saving to disk. Those five settings are useful when saving book formats out to fix with other tools. Unchecked, Replace space with underscores. Unchecked, Change paths to lower case. Format dates as: %b, %Y. File formats to save: all. I append calibre's book id (unique within that library) to title in the path to ensure that all books with the same authors and title such as multiple editions of the same book are saved out to the file system, rather than all but one being ignored by file system. The series information for file-name is consistent with my Metadata Plugboard templates, but I use Author_sort in the path to sort the folders by last name, and Authors rather than Author_sort in file-name to be consistent with the First_Name Last_Name convention I use in the Authors column, so when I Save To Disk books to be fixed then Add the fixed versions into calibre, if Add Books is set with Unchecked, "Read metadata from file contents rather than filename", I don't have to switch the Author name back from Last_Name, First_Name. I like prepending one zero to series for correct sorting in 2-digit large series and don't worry about sorting in rare cases of 3-digit large series. Saving Books template:
    Code:

    {author_sort}/{title} - ({id})/{authors} - {series}{series_index:0>2s| | - }{title}
  • Sending Books. Metadata management: Only on send. Format dates: %b, %Y. The Sending Books template is overridden where relevant by Metadata Plugboard templates for each device. So I set the Sending Books template to be identical to the Saving Books to Disk template for consistency, specifically for use with Connect to Folder (under the Sharing icon):
    Code:

    {author_sort}/{title} - ({id})/{authors} - {series}{series_index:0>2s| | - }{title}
  • Metadata Plugboards, for Kindle. Formats, MOBI. Device, Kindle2. Templates for Kindle to put metadata in relevant metadata fields inside the book format, to be read from there by device: prepend Series and Series Index into destination field Title:
    Code:

    {series}{series_index:0>2s| | - }{title}
    and Author Sort into destination field Author:
    Code:

    {author_sort}
  • Metadata Plugboards, for iPad. Set up for Connecting/Sharing, Connect to iTunes, by following instructions in the sticky (calibre + Apple iDevices: Start Here), with some exceptions and customizations. The books sent from calibre to iTunes get synced the next time I connect iPad or iPhone to computer/iTunes. I leave the Kindle chosen in Preferences, Conversion, Common Options, Page Setup, Output Profile, rather than changing Output Profile to iPad. I use a comma-separated #genres column rather than a text #genre column—this way doesn't necessarily send the correct primary genre, but avoids needing another column dedicated to primary genre, and I don't need iTunes to have the correct primary genre when I manage those iTunes books from within calibre. Plugboard settings: Formats, any format. Device, Apple. These templates put metadata in relevant metadata fields inside the book format, to be displayed in iTunes on computer or reader application on the iDevice—Series and Series Index prepended to Title into destination field Title:
    Code:

    {series}{series_index:0>2s| | - }{title}
    Author_Sort into destination field Author:
    Code:

    {author_sort}
    and one of several genres from my multi-value field #genres into destination field Tags (temporarily during this send) for iTunes genre:
    Code:

    {#genres}


Example Preferences, Sharing:
Spoiler:
  • Sharing by eMail. Set up and tested but seldom used.
  • Sharing over Net. Content Server set up, tested, seldom used. Connecting/Sharing, Connect to iTunes set up as discussed above in Import and Export, Metadata Plugboards.
  • Metadata Download. Downloaded metadata fields: Comments, Published date, Publisher. Sources checked and configured: Amazon (1; Comments, Published date, Publisher), Fantastic Fiction (1; Comments, Published date, Publisher), ISBNdb (1; Comments, Publisher), Open Library (3, sometimes useful for cover only). Occasionally for testing purposes or if one of the usual sources is down, I temporarily change fields or sources.


Example Preferences, Advanced:
Spoiler:
  • Plugins. Discussed below in the "Customization" section.
  • Tweaks, Author sort name algorithm. I use the default tweak for author_sort, to copy if there is a comma in the name, otherwise invert: author_sort_copy_method = 'comma'.
  • Tweaks, Control How Dates are Displayed These save horizontal display space in booklist.
  • Modified column (unused and kept hidden): gui_last_modified_display_format = 'yyMMdd'.
  • Published column: gui_pubdate_display_format = 'yyyy'.
  • Date column: gui_timestamp_display_format = 'yyMMdd'.
  • Tweaks, Control Sorting of Titles and Series in Library Display. title_series_sorting = 'strictly_alphabetic'—this is for consistency in sorting, to match the convention used by most sites that I refer to for correcting and standardizing metadata.
  • Tweaks, Control Formatting of Titles and Series in Templates. save_template_title_series_sorting = 'strictly_alphabetic'—for consistency.
  • Tweaks, Order of custom columns in Edit Metadata. For consistency in workflow and data entry, I set this to be congruent with my preferred order of custom columns shown in the booklist.
  • Tweaks, All Other Tweaks. All other tweaks are default.
  • Miscellaneous. Max simultaneous conversion/news download jobs: 3 — since I have an Intel quad core. Checked, Limit max simultaneous jobs to available cores. Abort conversion jobs that take more than: Never abort. Regarding Command Line Tools, I've rarely needed to use them, and am more oriented to Graphical User Interface than Command Line, so don't bother to install them unless I need them.
  • Keyboard. Command I, for third-party plugin Search the Internet to bring up browser on page of selected author at Internet Speculative Fiction Database (ISFDB). Others all default.
  • Template Functions. Default. Not used.



Customization
Spoiler:

Plugins:
Spoiler:
  • Plugin Functionality. The calibre application's functionality is primarily based in various plugins. Default plugins are part of the calibre download. Optional calibre-approved third-party plugins are available through MobileRead and calibre. Other third-party plugins that are not approved by the calibre developers may be available from other sites. Also, users who have some familiarity with programming in Python can create their own plugins.
  • Examples, My Third-Party Plugins. I use these third-party plugins available through calibre and MobileRead:
  • User interface action plugins: Count Pages, Find Duplicates, Extract ISBN, Modify ePub, Open With, Quality Check, Search the Internet.
  • Metadata source plugins: Fantastic Fiction, Barnes & Noble, Goodreads. (The last two are usually turned off, but sometimes used to supplement other sources.)
  • Conversion input plugin: DOCX Input. (New in June 2012, testing it.)


Libraries:
Spoiler:
  • Separate Libraries. Separate libraries might be suitable for different needs such as: testing versus stable, shared versus not shared, improving performance by splitting a huge library, or sets of books with different types of content. Some examples of types of content that might be managed in different library structures are: periodicals, comic books, technical papers, technical books and textbooks, fan-fiction, and fiction or popular non-fiction.
  • Number of Libraries. Minimizing the number of libraries avoids various convolutions, extra work, and associated headaches.
  • Custom columns for different types of content can be managed well within one primary library using the third-party plugin View Manager for different saved column views and restrictions based on tags.
  • Restrictions provide some of the benefits of separate libraries—both support simpler search criteria against a limited set of books and limit Tag Browser metadata lists.
  • The more libraries, the more work to implement common custom columns or metadata structure changes across libraries.
  • The features Create Catalog, Search, Edit Metadata, Content Server, and Booklist View of library are limited to books within one library at a time.
  • Copying/pasting metadata across different libraries is not quick or practical.
  • Comparing library booklists at a glance between two libraries simultaneously requires awkward solutions such as two computers side by side with extreme caution not to mess up a library's file permissions and database with changes written by different instances of calibre to any one library.
  • Comparing a library booklist to a catalog of another library in ebook form on a device is useful but somewhat limited.
  • Example, Primary Library. In my case I read mostly fiction, so I use one primary library and include small quantities of other types of content in the same library without needing tailored custom columns for various types of content. I also process newly added books in this primary library, completely processing them right away after adding to keep the library pristine.


Columns:
Spoiler:
  • Adding Custom Columns. Custom columns are specific to each library where created, except when creating a new library with the option checked to copy structure from the current library. I try to keep custom columns few and simple. The more metadata kept, or the more complex the automation with composite columns built from other columns, the more work later to maintain metadata consistently or to change library structure. Also, columns built from other columns may slow down performance in large libraries.
  • Examples, My Custom Columns:
  • Formats. Built from other columns, to see a book's formats at a glance in booklist.
  • FQ. Comma separated text, for Format Quality tags, including a format quality rating and any format problem tags.
  • Pages. Integers, format for numbers {0:,}, for page count by third-party plugin Count Pages.
  • Genres. Comma separated text, for primary genre and secondary genres.
  • Kinds. Comma separated text, for kind tags such as anthology, collection, omnibus, comic, magazine.
  • Status. Comma separated text, for status tags indicating: "to read", "my content rating", "series is multi-author", "own paper book", "no ISBN available", etc.
  • Awards. Comma separated text, for awards.
  • Note. Text, for any notes or additional information.
  • ISBN. Built from other columns, to see at a glance in booklist.
  • Source. Text, for the source such as name of vender or download site.
  • Updated. Integers (such as 120710 as yyMMdd), for the last date that the author was checked on a good internet metadata site and then Updated with any new books, that date entered in bulk for all books by that author. This is helpful in a sort or search to quickly determine when to do another update for what authors. The column type is integers rather than date for easier editing.
  • Sharing With Family. At present I am the only person in my household using calibre and ebooks. If there were multiple people in my household all using the same shared calibre library, I would handle it with a Read Status custom column (comma separated tags for personal rating and "To Read" tags) for each person, supplemented by each person using the third-party plugin View Manager to save preferred column order.
  • Hiding Unused Columns. Hiding columns that are unused saves horizontal display space in the booklist. I hide Languages, Modified, and Rating. Languages is hidden because all my books are in English. Modified is hidden because I don't ever refer to it. Rating is hidden because I value my own rating of book content more than an outside and potentially skewed popularity rating, particularly when many of the books I read are not popular.
  • Adjusting the Order of Columns. The order of columns in the booklist and on the Edit Metadata form facilitates ease of workflow and data entry. The third-party plugin View Manager can help by quickly switching views of the booklist to various saved column configurations. Column order in the booklist can be adjusted by dragging the column header to a new place, or by changing the order in Preferences, Interface, Add your own columns. The "order of custom columns in Edit Metadata" can be adjusted in Preferences, Advanced, Tweaks.
  • Examples, Order of Columns. This my column order in the booklist: Authors, Series, Title, Published, Formats, FQ (Format Quality), Pages, Size, Genres, Kinds, Status, Awards, Note, Publisher, ISBN, Date, Source, Updated, Tags. I usually keep the Note and Tags columns each less than 0.5 inches wide. For most activities I don't have to adjust column widths and can see all columns at once on the booklist. To maximize horizontal space I set the User Interface layout to Narrow (Preferences, Interface, Look and Feel) and usually keep the Tag Browser hidden (tag button at bottom right). To keep it simple, I wanted to familiarize myself with only one column view in the booklist so decided not to use the third-party plugin View Manager—now I'm accustomed to using one static column order in the booklist for many purposes. I also used the tweak to adjust the order of custom columns in Edit Metadata, to be congruent with custom column order in the booklist.


Tags:
Spoiler:
  • Using Tags. "Tags" are metadata entries that identify something about the book. They can reside in the Tags column or in any other columns. A tag can be a phrase, a word, an abbreviation, a number, and so on. Tags can be automatically downloaded from external sources into the Tags column using the "Download metadata and covers" command, per preferences set under "Metadata download". Tags also can be created and assigned according to a user's own tag scheme. I use my own tag scheme because I like its consistency and flexibility better than most tags automatically fetched from external sources.
  • Using Abbreviations. Generally using abbreviations for tags is not recommended because an abbreviation is not as human-readable as the full word or phrase. Personally I prefer abbreviations for several reasons. Abbreviations are faster for data entry. They require less horizontal space on display allowing me to see many more tags at a glance. In my case currently no other humans have to figure out what my tags mean, and the meaning of most of the abbreviations are obvious in themselves or in context with that column name and that book's author/series/title anyway.
  • Using Prefix Symbol for Multi-value Sorting. For comma separated multi-value columns, sometimes it helps to prefix a tag with a symbol such as exclamation point or underscore, neither of which conflicts with any other calibre functions. Doing this will sort the that "primary" tag to the left of any "secondary" tags in the displayed column. I use the exclamation point prefix for primary genre, format quality rating, "my content rating / has been read", "to read", and "newly added" tags.
  • Examples, My Tag Scheme:
  • Format quality rating, primary (#fq): !q0, !q1, !q2, !q3, !q4, !q5. From placeholder !q0 through excellent !q5. See "Assessing Formats" section for more discussion.
  • Format problems, secondary (#fq): Problem abbreviations such as "Hdr" (header/footer), "splP" (split paragraphs), "Pjam" (too-large paragraphs jamming together successive dialog by different people or other multiple should-have-been-separate paragraphs into one).
  • Genres, primary (#genres): !adv, !fn, !hr, !main, !mys, !nf, !pn, !ro, !sf, !thr, !uf. For convenience I separate into specific primary genres !sf (science fiction), !fn (fantasy), !uf (urban fantasy), and !pn (paranormal) because most of my reading consists of them. I use !nf (non-fiction) as a primary genre because I don't read a lot of it.
  • Genres, secondary (#genres): Such as vmp (vampire), dgn (dragon), yng (for young adult or child), clsc (classic), bio (biography), cmp (computer), etc.
  • Kinds, secondary (#kinds): an (anthology), shr (short story, novella, novelette), om (omnibus), etc.
  • Awards, secondary (#awards): h (hugo), n (nebula), wfa (world fantasy award), etc.
  • Status, primary (#status): !TR (to read), !r1 through !r5 (my content rating also indicating I previously read it). May have both a rating and !TR if I want to read it again.
  • Status, secondary (#status): m (multi-author series), pb (own paper book), xISBN (no ISBN available).
  • Tags, primary (#status): !new (newly added—tag automatically assigned by calibre during Add Books, per Preferences/Add Books.)


Searches and Restrictions:
Spoiler:
  • Learning About Searches. I used calibre for eight months before finally deciding to wean myself off the tag browser and directly use the the Quick Search box in the search bar above the booklist. Now I rely on the search box and usually keep the Tag Browser hidden. Using the search box is faster and more convenient. See the calibre User Manual, Search Interface and the following section calibre User Manual, Saving Searches.
  • Non-Prefixed Searches. Non-prefixed searches are useful in smaller libraries or restrictions, in conjunction with tags that are relatively unique rather than commonly used words, where that search string matches anything in the metadata per preference settings for search terms without prefixes—for example in the search box type "xanth" (no quotes).
  • Prefixed Searches. Searches prefixed by the column name provide results significantly faster in large libraries than non-prefixed searches—for example in the search box type "series:xanth" (no quotes). I usually use prefixed searches in the search box rather than non-prefixed searches.
  • Grouping Search Criteria. Search criteria can be grouped in parentheses and used with operators such as "or", "and", or "not". If a tag has internal space(s), enclose the tag with quotes.
  • Contains or Equals. When not using the equals sign in searches, by default calibre uses "contains" rather than "equals".
  • Saved Searches and Restrictions. Saved Searches or Restrictions are only available in the library where they were first saved, but creating a new library with the option checked to copy structure from Current Library will include any Saved Searches and Restrictions in the new empty library.
  • Saving Searches. To create a Saved Search, type the search criteria and operators into the search box. To the right of the search box, type a name for the search into the little menu box labeled Saved Searches. All the way to the right of that, click the little square button that's green with a white cross, to save the search. Click the down arrow on the Saved Searches menu box to see the menu of Saved Searches, and choose the new Saved Search.
  • Regular Expressions in Searches. Regular Expressions can be used in searches. Useful regular expressions can be collected, and each can be saved as a Saved Search, with the regular expression phrase preceded by a tilde to tell calibre that what follows is a regular expression. "Saved Searches" is one convenient place to save useful regular expressions for later recall, which helps someone like me who is a beginner at regular expression syntax. (Another place to save regular expressions is Edit Metadata in Bulk, Search and Replace, Search mode: Regular Expression). For an example of a regular expression in a Saved Search, see "Author 2 commas" in the list of examples below.
  • Deleting Searches. To delete a Saved Search, to the far right of the search box, click and hold the green button with the plus sign then choose from menu "Delete saved search".
  • Using Restrictions. A Restriction menu choice is created automatically when a Saved Search is created, and then deleted when a Saved Search is deleted. To apply a Restriction, select a Restriction name in the Restrict To menu box. Now the library is restricted to all books that meet the search criteria of the associated Saved Search. The Restriction is reflected in the lists of tags by Category in the Tag Browser, now limited to any tags for books meeting the search criteria of the Restriction. Searches or Bulk Metadata Edits done within a restriction are limited to only those books inside the restriction. Restrictions allow using simpler criteria for searches. Restrictions provide some of the benefits of separate libraries, within just one library.
  • Examples, Some Saved Searches/Restrictions:
  • New. Based on tags:!new where the "!new" was automatically assigned by calibre into the Tags column as books are Added to the library, per Preferences, Adding Books, "Tags to apply when adding books".
  • NonFiction. Based on: #genres:!nf or tags:Catalog or tags:News .
  • Fiction. Based on: not search:"=NonFiction" .
  • FQ Problems. Based on format quality rating tags: not (#fq:!q4 or #fq:!q5) .
  • To Load. For loading news, catalog, and "To Reads" onto devices: tags:Catalog or tags:News or #status:!TR .
  • No Cover. Based on: cover:false .
  • No ISBN. Based on: identifiers:=isbn:false .
  • Author 2 commas. To find any author whose name contains 2 commas, using a regular expression after the tilde:
    Code:

    author:"~.*,.*,"


Miscellaneous:
Spoiler:

Icons:
Wishlist Format:
  • Using "Empty Book" Wishlist Format. Optional. It may be useful to add an empty Wishlist Format to Empty Book records specifically for the purposes of (1) embedding metadata into a format so wishlist items can be transferred to a device which reads metadata from fields in the format, or (2) including wishlist items in future Save to Disk operations because Save to Disk does not include Empty Books without formats. I like having a convenient way to save or send out wishlist items along with "real" books. It's easy to exclude them from an operation when necessary.
  • Creating Wishlist Format. To create a Wishlist Format, open a text editor. Type "Wishlist Format" (or whatever) as the sole content. Save the file as text with filename "Wishlist Format". Drag and Drop the file onto calibre booklist window to add it. Enter "Wishlist Format" in Authors and Titles columns. Convert to Preferred Format, the same format chosen in Preferences, Interface, Behavior as Preferred Output Format. Save to Disk into a "Fix Books" working folder.
  • Adding Wishlist Format. In calibre after creating Empty Books, select the Empty Book wishlist records, under the Add Books icon choose "Add file to selected book records", and navigate the pop-up file-browser dialog box to point to that Wishlist Format in the "Fix Books" folder.



Getting Books
Spoiler:

Consolidating Books:
  • Consolidate Books. Initially consolidate books into as few places as possible to avoid having to search for books in many places just to know what is already owned. Having books in one primary calibre library makes it easy to search for books in calibre. Using one folder for "Pending Add to calibre" makes it easy to conduct operating system searches of that folder for any particular author or title. The books in the "Pending Add" folder can be left there then processed into calibre later one author at a time.
  • Consolidating from Libraries. To consolidate from other calibre libraries, from each library use the Copy To Library command to copy (delete after copy) selected books to the primary library. If these books were not completely processed yet, process them all now to keep the primary library pristine with processed books. Or as an alternative, save to disk any unprocessed books from calibre libraries to a folder in the "Pending Add" folder.
  • Consolidating from Devices. If any devices have books that are not already processed into the primary calibre library, transfer them to the primary library and process them all now to keep the primary library pristine with processed books. Or as an alternative, copy or save to disk any unprocessed books from devices into a folder in the Pending Add folder. Then delete the copies on the devices.
  • Consolidating from Folders. To consolidate from various non-calibre folders, move or copy all of those source folders into the Pending Add folder. It's better to move them than copy them to avoid duplication of books all over the computer.
Choices and Comparisons:
  • Learn What is New. Learn what is new on a site that has good metadata such as Fantastic Fiction. Fantastic Fiction has lists of new books released or forthcoming for the current month, the prior 3 months, and the next 3 months.
  • Choose Current Author. Choose an author. This author becomes the "Current Author" for further processing.
  • Compare with Books Owned. Determine what books are already owned for Current Author by doing an operating system search of the Pending Add folder for Current Author, and a calibre search in the primary calibre library for Current Author. Compare those lists with the list for Current Author on a Good Metadata site such as Fantastic Fiction or Internet Speculative Fiction Database (ISFDB). The third-party plugin Search the Internet is useful for opening the browser to the selected author's page on the Good Metadata site. Decide on what books to potentially get that are not already owned. If there is nothing to get for this author, go back a step and choose a different Current Author.
  • Choose a Source. Browse ebook vender/distribution sites to determine which internet sources to use. I'd rather buy all items from one source when possible to keep things simple and save time and effort. There are two ways to compare sources:
  • Get Books Feature. In calibre's Get Books feature, choose the desired sources, then search for Current Author. A list of books by that author at those sources displays. Sort by Store (source) to see all books there by that author, or sort by Title to compare prices for a title between sources. Get Books also shows which books have DRM and what formats are available at each source. Double-clicking a title brings up the book's page on the source site. It is easy to compare prices between sources this way, choose a source, then automatically open a browser window there. I use Get Books when comparing between sources is important.
  • Browser. Manually navigate by browser to a likely source site such as Amazon, Barnes and Noble, MobileRead, or Project Gutenberg. Or use the third-party plugin Search the Internet to get there. I use either method. Search for the author there if necessary. Repeat with other sources. It is not as easy to compare between sources this way.
Downloading from Source:
  • Download Books. Go to the chosen source site in a browser, if not already there. Using whatever methods the source site supports, download the desired books by the Current Author. Download to computer rather than direct to device.


Adding Books
Spoiler:

Pre-Assessing:
  • Pre-Assess Book Formats. If there is more than one book format for a title, from the operating system quickly pre-assess each format in a suitable reader for that format to find the best format to Add. To do this easily, in the operating system set the suitable Reader application for each file type, such as Stanza for EPUB, Kindle Previewer for MOBI or other Kindle formats, and Adobe Reader for PDF. This allows double-clicking a filename in the operating system to open it in the appropriate application.
Adding:
  • Review Preferences for Adding Books. The methods for Adding Books are dependent on relevant Preference Settings. Unchecked, Automerge—because I want to later assess any multiple formats or duplicates of book records to decide what to delete, and that is less confusing for formats in separate records. Also when the little dialog box comes up during the Add, I usually choose to add author/title duplicates so I can later assess them and decide what to delete. Unchecked, Swap author last and first name—as a personal preference, the swap is usually easier to do other ways after the Add than remembering to switch this option on and off in Preferences. Checking the checkbox for "Read metadata from file contents rather than filename"—is easiest and sometimes works well for new retail books in EPUB or MOBI formats, but may provide unpredictable results that have to be corrected. So I usually leave it Unchecked so calibre will use the regular expression in the Regular Expression menu box to guess metadata from the filename, which requires using one of these methods:
  • Before Adding, in Operating System. Standardize author, series, and title in the filename out in the Operating System first before Add Books. This can be done manually, or with a script that standardizes file names, or with successive passes with different regular expressions in a file renamer tool. The filename structure needs to match the chosen regular expression for Add Books by guessing from filename. I like this method best. Normally I add just a few books at a time, so it is usually easiest and fastest to edit the filenames manually.
  • During Adding. Successively Add Books in small batches, each batch's filename structure matching the chosen Add Books Regular Expression or a new Regular Expression written on the fly to suit that batch of books' different file naming convention, with the Regular Expression automatically putting the relevant groups into the correct calibre columns for author, series, and title. This one seems easiest and fastest for adding a large number of books quickly for someone with sophisticated Regular Expression skills, which I don't have. And I avoid adding a large number of books at once.
  • After Adding, in calibre. Standardize author, series, and title from the filename inside calibre after Add Books imports them all as a mess. Can be done manually with cut/paste one field at a time for one book at a time, or with multiple selections in Edit Metadata in Bulk, Search and Replace, Regular Expression mode to copy relevant metadata from one origin field (Search Field) to where it belongs (Destination Field), followed by another pass to delete the offending element from the origin field—so for each origin field that has an element to be moved, this requires a pass to copy to destination and a pass to delete from origin.
  • Add Books. In the operating system Downloads folder or Pending Add folder, select books to be added by Current Author. Drag and drop the selection from that folder onto the primary library's booklist window. If that Add method didn't work well, delete those books, reset preferences for Add Books to a different add method, and then Add Books again. I usually Add Books a few at a time, all from the same source and all by the same author. Note: dumping a lot of books into calibre at once is not advised, but if necessary, from within calibre under the Add Books icon, choose whichever of the top three menu commands best fits the way books are organized in that folder. For descriptions of each Add Books command see calibre User Manual, Graphical User Interface, Actions, Add Books.
Preliminary Editing of Metadata:
  • Enter Source Metadata. Select all newly added books from one source, Edit Metadata in Bulk, enter source data in the Source column. This is usually the name of the download site, the name of the source, or the name of the source folder for books that were already on computer. If there are any other books by Current Author not yet added, such as in the Pending Add or Downloads folder, go back a step and add them.
  • Edit Authors, Title, Series. If necessary, move metadata from Authors, Title, and Series into the appropriate columns. This can be done manually for each book record, or selectively using Edit Metadata in Bulk, Search and Replace, Regular Expression mode to put the relevant element into the correct calibre field for Authors, Series, or Title.
  • Create Wishlist Records. To create Wishlist records as placeholders or as a list of books to acquire in the future, under the Add Books icon choose Add Empty Book. For each, enter the Title, and if necessary enter or correct Author(s). For all of them, I enter the tag "!q0" in the custom column FQ (Format Quality). I also add a dummy wishlist format to wishlist records.


Assessing Formats
Spoiler:

Preferred Format:
  • Bulk Convert to Preferred Format. For all newly added books, convert to Preferred Format. That is the same format chosen in Preferences/Interface/Behavior as Preferred Output Format. Many calibre users choose either EPUB or MOBI. EPUB opens faster than other formats in the calibre viewer because calibre internally converts all other formats to EPUB to display in the viewer, which takes extra time. EPUB is also good for initial assessment, possible fixing, reading on e-ink devices supporting EPUB or tablets with ebook reader apps supporting EPUB. In my experience converting an EPUB master to MOBI for Kindle works slightly better than converting a MOBI master to EPUB. Better tools exist for editing EPUB than for editing MOBI or other Amazon formats.
  • Example. I use EPUB as my Preferred Format.
Assessment:
  • Assess Preferred Format. Open Preferred Format in calibre viewer. Examine title and copyright pages. If necessary, correct Authors, Series, and Title. Assess the format by scrolling or paging rapidly through the book format, looking for format problems. For minor problems such as line spacing that are fixable through conversion, reconvert using relevant conversion settings, then re-assess.
  • Assess Other Available Formats: If the Preferred Format has format problems, assess any other available formats for the same book. If other formats exist outside calibre, pre-assess them in the operating system, choose the best, and Add to calibre. For each other format for that book in calibre, use the third-party plugin Open With to open the format in the appropriate assessing tool, such as Acrobat for PDF or Kindle Previewer for MOBI. After assessing each format choose the best format with the least serious problems, convert it to Preferred Format, and re-assess. For minor problems fixable through conversion such as line spacing, convert the best format to Preferred Format using relevant conversion settings, then re-assess. The best incoming format becomes the master copy for all further conversions. If satisfied with the Preferred Format, to avoid any confusion later, keep Preferred Format and remove all other formats for that book except problem PDFs, in which cases PDF becomes the master copy for reading in PDF rather than EPUB or MOBI format. If unsatisfied, don't remove the other format yet because it may be necessary later during the process of fixing books.
  • Enter Format Quality Tags. In the custom column FQ (Format Quality), enter a tag for each type of format problem found and not yet fixed (such as Header/Footer),
  • Enter Format Quality Rating. Enter the applicable format quality rating. I use this as a basis for comparison between any duplicates, and also as a method to identify which books (formats with lower quality ratings) should be upgraded with better formats in the future. Some calibre users employ a column for retail quality versus not retail quality. Using format quality ratings is similar but more flexible and fine-grained.
  • Examples, my Format Quality Ratings:
  • !q0 is a wishlist item. Wishlist item means an Empty Book (with or without an empty book wishlist format) as well as any book where the only formats available are fixable only with a lot of manual work, which isn't worth the time expense to me. I use "!q0" as the basis to color text red in specific columns in the booklist row. When using catalogs or content server with devices, "!q0" indicates wishlist items.
  • !q1 is for rare cases to delay fixing until later, so don't delete Original Format. I rarely use "!q1" because I tend to fix everything necessary immediately after adding, or will label a format "!q0" if a quick fix isn't possible.
  • !q2 is for rare cases of more than minor annoyance but not easy or worthwhile to fix in Preferred Format, so don't delete Original Format. Usually applies to a PDF that is more readable in PDF format than Preferred Format due to complex layout, graphics, and/or technical content.
  • !q3 is okay, readable with only minor annoyance. Keep only Preferred Format.
  • !q4 is good, readable with no annoyance. Keep only Preferred Format.
  • !q5 is excellent, no annoyance, excellent Table of Contents, excellent layout, no problems discovered at all. Keep only Preferred Format. Do not assign "!q5" until after reading a book, in which case it replaces the initially entered "!q4". I rarely use "!q5" because "excellence" is a moving target, less concrete than "no annoyance". As I learn more about ebook formatting my standards for excellence gradually get higher, while my annoyance level for trying to read past any particular format problem tends to remain unchanged.


Fixing Formats
Spoiler:

Preliminaries:
  • Review Preferences for Save and Add. Review preferences for Saving Books, and for Adding Books. Reset them as desired if necessary. I use the settings described in the "Preferences" section.
  • Avoid Extremes. The overall goal as a reader isn't a perfect book, but to spend the least amount of time to make it readable with as little annoyance as possible. There are two extremes regarding fixing format problems. One is to just want to read books and not care much about the formatting and any format problems. The other is to want to make each format as perfect as possible. I try to sit the fence between extremes. I don't worry about all format problems, just the ones that annoy me the most now that I currently know how to fix. If I can't fix it quickly, I call it a wishlist item, hope to replace it with a better format in future, and go on to the next book to fix.
For Each Book to Fix:
  • Choose Method(s) for the Fix. The simplest method is to use an Editor to fix all problems in a book. Optionally, various fix methods can be used singly or in combination for each type of problem in a book, using the method or tool best suited for that type of problem. Alternate methods within calibre include doing a conversion using various calibre conversion options such as Search/Replace, or the third-party plugin Modify ePub for minor fixes. As someone relatively new to fixing ebook formats, I prefer to address various minor problems using conversion settings or Modify ePub when I can. Where that doesn't work or is insufficient, I use an Editing application to fix problems.
  • Choose an Output Format and Editor. Choose an output format that can be opened by an Editing application. Choose the Editing application. Examples:
  • RTF for fix in OpenOffice, Word, or other editor. Note that calibre does not support DOCX (Word) or ODT (OpenOffice) as output formats.
  • EPUB for fix in Sigil.
  • HTMLZ for fix in any HTML editor.
  • PDF for fix in Acrobat Professional (not the Reader) or another PDF editor.
  • MOBI for fix using various MOBI or Amazon tools.
  • TXT or TXTZ for fix in OpenOffice, Word, or other editor, usually loses text formatting such as bold and italic.
  • Choose Conversion/Fix Sequence. Minimizing the number of conversions in a conversion/fix sequence saves time, simplifies workflow, and achieves higher quality of format—because each conversion loses more formatting and content information while introducing more errors, like successively photocopying copies of copies. I've tried many sequences. Sigil uses the fewest conversion steps if Preferred Format is EPUB, and is more ebook friendly than editing HTML directly in an HTML editor, but also requires some familiarity with EPUB structure and skill with HTML. Many ebook professionals and advanced users use Sigil. Examples of Conversion/Fix Sequences:
  • Original Format —> RTF, Word fix, save as HTML, zip —> Preferred Format. I use this.
  • Original Format —> RTF, OpenOffice fix, save as HTML, zip —> Preferred Format.
  • Original Format —> EPUB, Sigil fix, save as EPUB —> Preferred Format if not EPUB.
  • Original Format —> HTMLZ, unzip, HTML Editor fix, zip —> Preferred Format.
  • Convert To Fixable Format. Convert if necessary to choice of fixable format. For this example I choose to use the "RTF, Word fix" sequence, so convert to RTF.
  • Save To Disk. Select the book. Save To Disk into a Fix Formats folder.
  • Open the Format in Editor. Drag and drop the saved RTF onto the Editor icon.
  • Replace Each Problem with Fix. For each problem, choose options in Find & Replace carefully. Search for the problem, put the fix in the Replace With box, test the Replace individually a few times, then Replace all. Get rid of headers, footers, and page numbers while avoiding splitting paragraphs. Put Chapter Headings in appropriate style. Fix any other annoyances that are quickly fixable. For more information see Ebook Formatting Tutorial, by amalthea. Also see the Ebook Conversion section of the calibre User Manual, including the subsection on "Format-specific tips".
  • Save as HTML. Save as HTML (filtered) into its own new folder. Depending on the Word version and platform, the choices and options may look different. Choose the option "filtered" if available. In Word:Mac 2011, I use the save choice "Web Archive (.htm)" with the option "Save only display information into HTML" which has better results than "Save entire into HTML."
  • Zip. Zip all the files contained in that folder. Rename the ZIP file if necessary by copying/pasting from the filename of the unfixed format originally saved out from calibre.
  • Add into calibre. Drag and drop the ZIP into calibre.
  • Convert to Preferred Format. Convert the ZIP to Preferred Format.
  • Re-Assess. Do quick assessment of that new Preferred Format. If it doesn't look okay, either convert the ZIP to Preferred Format with different conversion settings to fix it, or go back to the Fix Formats folder, restart with the RTF in Word and fix it better. In the FQ column: update the Format Quality Rating, remove any problem tags for problems that were fixed, and enter tags for any newly discovered problems that weren't fixed.
  • Remove Unfixed Duplicate. Compare metadata in fixed book record with corresponding unfixed book record. If necessary correct metadata for the fixed book. Remove the corresponding unfixed book.


Editing Metadata
Spoiler:

For All Newly Added Books of Current Author:
  • Apply Author Naming Convention to Author. Apply preferred convention of "FN LN" (First_Name Last_Name) or "LN, FN" (Last_Name, First_Name). I use the convention FN LN primarily because it is less work, and helpful when using internet sites to get books or standardize metadata—the sites I use want the FN LN convention pasted into the site's Search box or used automatically by an author template. For specific information on switching author name conventions including handling author sort issues, see calibre User Manual, FAQ, How does calibre manage author names and sorting? My "author_sort name algorithm" tweak is set to 'comma'. Note that doing a swap to apply a different convention may adversely affect author or author_sort values, for example for author where last name is followed by "Jr", "Sr", "Esq" or other suffixes, or last name is preceded by "de la", "van", "le" or other prefixes. So such problems will need to be fixed later. Methods for changing the author name convention include:
  • Quality Check. Select all books to be switched, use third-party plugin Quality Check, Fix menu, Swap Author command. This is the easiest method.
  • Bulk Edit Author. Select all books by one author. In Edit Metadata in Bulk, Author(s), change the author name into the desired convention by typing the entire name as desired, or by copying/pasting parts of it to other parts of it.
  • Regular Expressions. Select all books to be switched, Edit Metadata in Bulk, Search and Replace Tab. Choose appropriate Regular Expression in Search Mode menu box. Choose appropriate Regular Expression in the Load Search/Replace menu box, click OK. If the Load Search/Replace menu box does not yet have the necessary Regular Expressions saved, create them as follows:
Spoiler:
  • Select two books that both have the same naming convention "FN LN".
  • In Edit Metadata in Bulk, Search/Replace Tab, choose "Regular Expression" in the Search Mode menu box.
  • Choose "Replace Field" in the Mode menu box and next to that check the Split Result checkbox.
  • Keep Case Sensitive checked.
  • Leave blank the menu box for Apply Function After Replace.
  • Choose "authors" in the Search field menu box and also choose "authors" in the Destination field menu box.
  • In the "Search For" and "Replace With" boxes, enter the Regular Expressions below for "FN LN --> LN, FN" into each relevant box, then check results in the Test Results boxes at the bottom.
  • Click the Save button next to Load Search/Replace menu box and Name it "FN LN --> LN, FN" in the little dialog box that pops up.
  • Click the Apply button.
  • Again, choose "authors" in the Search field menu box and also choose "authors" in the Destination field menu box.
  • Enter the Regular Expressions below for "LN, FN --> FN LN" into each relevant box, then check results in the Test Results boxes at the bottom.
  • Click the Save button next to Load Search/Replace menu box and Name it "LN, FN --> FN LN" in the little dialog box that pops up.
  • Click OK button.

FN LN --> LN, FN

Search for:
Code:

(.*) (.+)
Replace with:
Code:

\2, \1
LN, FN --> FN LN

Search for:
Code:

(.+), (.*)
Replace with:
Code:

\2 \1
  • Add Wishlist Items. Optionally, create wishlist records (empty book, or empty book containing a dummy wishlist format) for any omnibus, collection, or anthology elements that would be nice to have separately with easily searchable metadata, or would be good placeholders for series numbering. For each wishlist item, enter or correct metadata in Authors, and enter Title. For all of the wishlist items, do a Metadata Download and enter "!q0" in FQ (Format Quality). Creating wishlist records for elements of omnibuses at this point simplifies correcting series information a little later. As a personal preference, I do this only for omnibus elements that are novels, and not for short story elements in omnibuses, collections, or anthologies.
  • Count Pages. Count pages using the third-party plugin Count Pages. I don't exclude wishlist items with dummy wishlist formats.
  • Extract ISBN. Optional. To extract ISBN, use the thiird-party plugin Extract ISBN. The ISBN makes the Metadata Download more precise, but it also limits the choices in Metadata Download so not using the ISBN may be better in some cases. I usually don't use it unless there was a problem with Metadata Download without it.
  • Download Metadata and Covers. If unsure, first review the Preferences for Metadata Download to make sure only the desired options are checked, to be certain not to overwrite any of the columns previously edited. Then download the desired metadata and covers. My download choices usually are: Published date, Publisher, Comments (Cover and ISBN are automatic.) I keep only a few sources checked, figuring the more checked, the slower the grab. I use Amazon and (third-party plugin) Fantastic Fiction. I also use ISBNdb and Open Library. I keep other metadata sources unchecked and use on a case by case basis when needed.
  • Enter Other Tags. For each newly added book including wishlist items, enter tags in remaining columns such as Series, Genres, Kinds, Status, Awards, Note. These are first pass entries that may be corrected later.
For All Books of Current Author:
  • Remove Duplicates. Check manually for duplicates across all newly added books and previously processed books by Current Author. For each set of duplicates, ensure they are duplicates, if necessary correct metadata for whichever book has the better format quality rating, then remove the books(s) with worse format quality. The third-party plugin Find Duplicates applies to the entire library, so I delay using it until later.
  • Compare Metadata with Good Metadata Site. Compare metadata of all books by Current Author with the metadata on a site that has good metadata. To make the metadata as consistent as possible, it's important to use a site that has good lists of consistent metadata and good metadata conventions across multiple authors and series, such as Internet Speculative Fiction Database or Fantastic Fiction. I do that by using the third-party plugin Search the Internet or double-clicking on Authors in the Book Details panel (per the author template in Preferences, Look and Feel, Book Details). Other potential sites: Goodreads is okay; Wikipedia is sometimes okay for popular authors; individual author sites are sometimes good, usually okay, sometimes bad; Amazon or other vendor sites are usually bad regarding standardization of series name and index.
  • Edit Metadata For Each Book:
  • Authors. Ensure all co-authors are entered and are in correct order. If necessary correct prefixes to author last name such as "de la" or "van". Ensure author initials are each followed by a dot. If last name has a suffix such as "Jr", "Sr", or "III", I copy the suffix to the Note column as for example "au: Jr" and delete the suffix (and any comma before it or dot after it) from Author. Removing the suffix from Author avoids author_sort confusion when author_sort tweak is based on comma. Removing the suffix and using dots after initials also avoids trouble with certain Good Metadata sites wanting or not wanting a comma before the suffix or dots after the initials when directly copying/pasting to the search box of the site, or using the third-party plugin Search the Internet for author. Removing the suffix is a simple solution that avoids needing to implement more complex solutions for the author link templates, which still might not solve all cases for all sites.
  • Author_sort. If Author contains prefixes to author last name such as "de la" or "van" that the author uses as part of the last name, change the author sort entry for that specific author in Manage Authors (control click Authors category in Tag Browser, Manage Authors) so that the author will sort by prefix and last name rather than just by last name. For example, I want author_sort to be "de la Cruz, Melissa" rather than "Cruz, Melissa de la". Or I want "van Vogt, A. E." rather than "Vogt, A. E. van".
  • Series and Series Index. Enter or correct Series name and Series Index using the same Series Name and Series Index used by the Good Metadata Site. This is important particularly for multi-author series where the series name and indexing should be consistent across authors. Optionally, when a single-author series is complete and not likely to be added to, change the series name to the broadest available (such as Valdemar rather than Mage Storms) and Index order to reading order by internal content rather than publication date, by using a site such as the author's official site that lists the broader Series' reading order. Otherwise, try to use just one convention for Series and just one convention for Series Index consistently across all authors and series in the library. I also give individual books that are not in a series the Series Name "(1)" and Series Index "1" because otherwise later when I notice series is blank I have a hard time knowing if I checked to see if that book was in a series.
  • Title. Double-check Title. Append edition information in parentheses after title, such as "(3rd Ed)".
  • Published Date. The Published Date provided through Metadata Download often needs to be corrected. I change this to date of first publication for most fiction or popular non-fiction, or to specific edition date when that is important such as for technical books.
  • Cover. Paste over bad or missing cover with good cover copied from Good Metadata Site. I don't worry about any other covers that might already be inside the format, and don't bother to remove those later. For any short stories or other books with no covers existing or available, generate a calibre cover. A calibre cover means a cover was looked for but didn't exist or wasn't found at the time the book was processed.
  • FQ (Format Quality). Ensure each book has a format quality rating, including wishlist items. Ensure the format quality rating is appropriate to any format problem tags that were previously entered.
  • Formats. Ensure each book has a Preferred Format. I include wishlist items in this. If a wishlist item has no format, I select it and use Adding Books "Add files to selected book records" to add a dummy Wishlist Format to it. For any book that has more than one format, ensure the kept Original Formats are for format problems by examining format quality tags. For books with format quality rating of "!q1" (fix later) or "!q2" (read Original Format) keep the Original Format as well as the Preferred Format. For books with other format quality ratings, remove all formats except Preferred Format.
  • Pages. Ensure each book has a Page Count.
  • Genres. Ensure each book has a primary genre and any relevant secondary genres. Enter any desired secondary genre tags to track things such as young adult, vampire, magic, alternative history, military, and so on.
  • Kinds. Enter or correct tags for collection, anthology, omnibus, short story, magazine, etc.
  • Status. Enter or correct tags to indicate if it is a multiple-author series, if it's a "To Read", if paper book is owned, and (when memory serves) "My Content Rating" for any unrated books that were previously read. For meta-metadata tags that apply at above book level at author or series level, ensure it's applied to all relevant books at that level.
  • Awards. Ensure any awards are entered. Enter awards. I only track awards for speculative fiction and paranormal genres, only for novels, and only with prestigious awards like Hugo, Nebula, World Fantasy, Pearl, and RITA.
  • Note. If any books have or need a note or other tag in Note column, ensure it's applied to all relevant books by that author or in that series.
  • Publisher. Metadata Downloads bring in variations of names for Publisher, where some may be Publisher Name, Inc., Publisher Name Trade, Publisher Name Enterprises, Publisher Name Fantasy, Publisher Name Science Fiction, etc, all referring to one publisher. I standardize those to the simplest name that identifies the publisher. Such as Random House rather than Random House Inc or Random House Trade, or Tor rather than Tor Science Fiction or Tor Fantasy.
  • ISBN. Ensure each book has an ISBN unless an ISBN doesn't exist for it. In those cases I tag them "xISBN" in Status column. Convert any remaining ISBN10s to ISBN13s by using the web app ISBN Converter.
  • Source. Ensure Source indicates source name. If missing, enter source name or "Unknown".
  • Tags. Remove any temporary tags from the Tags column, including the "Newly Added" tag that calibre automatically assigned in Tags column and any other temporary processing tags that may have been used for convenience while processing this book.
  • Enter Date Updated. Edit Metadata in Bulk for all of Current Author's books, enter today's date as the date this author was last updated.
  • Delete Current Author in Pending Folder. Now that all currently owned books by Current Author have been processed, conduct an operating system search of the "Pending Add to calibre" for any books by Current Author, ensure they were Added, and delete those that were Added. Do the same with the Downloads folder.


Reading Books
Spoiler:

In calibre Library Booklist:
  • Generate Catalog. Optional. Select all books in the library and generate a catalog in the form of an ebook. The configuration I usually choose for generating catalogs is: In the Catalog dialog box, Catalog Options tab, select catalog format EPUB. In the ebook Options tab, check only "Books by Authors". Also in the ebook Options tab, in the "Read Books" area select FQ (Format Quality) column and enter "!q0" for the value, which will use checkmarks in catalog to indicate wishlist items rather than books read. I don't use catalogs often. In the past when I had multiple libraries, catalogs were useful as an easy way to visually compare between a Library 1 catalog on device and a Library 2 calibre booklist on computer.
  • Search Library for "To Reads". Search for the tag used for "To Read" to get a list of those books. Add any newsfeeds or catalogs to the list if they are not sent automatically to devices. I use a Saved Search named "To Load" to find a list of all the items I might want to load on devices, including "To Reads", News, and Catalogs. I like to manually control what items are sent to which device. Another method for managing "To Reads" is to use the third-party plugin Reading List, for managing reading lists and syncing those with devices.
  • Load Device with Connect To iTunes. Select books to load through iTunes, then Choose Connect To iTunes under the Share icon. Click Send to Device icon. The only exceptions are problem, graphic-laden, or technical PDFs that are better to read in PDF format, so Send any of those by choosing the menu command to send the specific format. Later connect the iPad or other IOS device by USB cable and sync through iTunes. I load my iPad this way.
  • Load Device with Send To Device. Connect a device such as Kindle. Select books to load, then choose Send To Device. Say yes in dialog box for calibre to automatically convert them if necessary. After loading, disconnect device. I load my Kindles this way.
  • Example, Choosing Between Devices. I do most reading on an e-ink Kindle rather than iPad since I read mostly fiction and most of that fiction is text except for cover and sometimes a map or two. The Kindle is lighter than the iPad, has less glare, and the battery lasts longer. On Kindle I nearly always have wireless turned off so the battery needs charging less frequently. I prefer to read complex, graphic-laden, or technical PDFs on iPad.
  • Example, Loading Devices. I load a device only a few books at a time, usually less than ten. I usually load using USB cable rather than wirelessly. When I travel on a long trip, I bring along a laptop holding the calibre library as well as an e-ink Kindle. For short excursions out of the house I often bring the Kindle, and won't have time to read more than one or two books. Loading by USB cable is simple and cancels the need to rely on a cloud, the Content Server or any awkward connection or selection convolutions with the device's browser or a reader app for wireless downloading. Regarding clouds (Amazon Cloud, iCloud, Dropbox, Sugarsync, etc), which cloud has which books?—I buy books from various sources, and at this point I don't want to bother with syncing all of my books across all of my devices by putting my entire calibre library onto a cloud and having to pay for extra storage space.
  • Note, Collections. I don't use collections on devices or in calibre columns, because I find it easier to refer to tags in calibre while avoiding extra work or extra columns devoted to unnecessary redundancy. Since I load reading devices with only a few books, further organization of them there isn't necessary. The third-party plugin Kindle Collections works well for Kindle users who want to use collections in calibre and on Kindle.
On Device:
  • Read Some Books. Read some books on a reading device. This is the reward for all the hard work.
In calibre Library Booklist, For Each Book Read:
  • Rate Content Quality. Soon after reading each book, rate that book's content quality. Examples: I enter this tag in the custom column Status. I prefer to use my own tags, rather than a yes/no "Been Read?" column plus the default Rating stars column. Using "!r1" through "!r5" takes up less horizontal display space and indicates my content rating as well as that the book was read, easily visible sorted to the left in a multi-tag column.
  • Remove "To Read" Tag. Remove the tag "!TR" from Status column.
  • Revise Format Quality Tags. In cases where more serious format problems were discovered, try to fix them right away, or change the Format Quality Rating to "!q0" to downgrade to wishlist/placeholder, and enter tags for the type of format problems discovered while reading it.
  • Revise Other Metadata. Correct any other metadata as necessary, such as secondary genres, with the new knowledge about the book gained from reading it.
  • Remove Format. If a format (such as MOBI) was converted from Preferred Format (such as EPUB) just for reading on that device, remove that device-specific format from the book record in the library.
In calibre Device Booklist, For All Books Read:
  • Remove Books from Device. Switch to the device booklist and remove previously read books from the device.


Maintenance
Spoiler:

For Each Library:
  • Remove Duplicates. Periodically use the third-party plugin Find Duplicates to find duplicate books within a library. Do a pass for Similar Titles and Authors, a pass for Fuzzy Titles and Authors, a pass for ISBNs. For each set of duplicates, ensure they are duplicates, if necessary correct metadata for whichever duplicate has the better format quality rating, then remove the books(s) with worse format quality.
  • Do Quality Check. Optional. Periodically use the third-party plugin Quality Check to check various things in a library's metadata and formats.
  • Check Library. Every few months for each calibre Library, under the Library icon select the Library Maintenance command Check Library. Click Yes when the dialog box asks to check if the files in library match information in the database. If calibre reports any problems, look in the operating system in the calibre library folder at each problem to ensure each problem is okay to delete, but don't change or delete files there. To delete the problem files, in the Check Library dialog box put a check in those problem items' checkboxes to mark them, click the Delete Marked button, click the Done button.
  • Ignore calibre Library Folders and Files. Treat each calibre library folder, subfolder, and their contents like a "black box" that will shock or burn if changed directly out in the operating system. Do not make any changes to them except indirectly through editing the Author or Title of a book from within calibre, moving an entire Library from within calibre, or using other calibre commands. For further discussion, see: Want to change the folder structure of the Calibre library?, sticky by Manichean and Worldwalker.
  • Check, if Moved. If the library was moved recently, ensure that automatic backups and antivirus scan exclusions include the new location. (See details below under "For Computer.")

For calibre Application and Plugins:
  • Download and Install calibre Updates. Download calibre only from the official calibre site. When an update is available, a message "Update Found" with version number appears on the bottom right of the calibre window. Clicking that brings up the official download page. Updates of calibre are usually released every Friday with new capabilities and bug fixes. I like to update right away and review the "What's New Changelog" while downloading. Sometimes the changes provide opportunities to improve methods or workflow.
  • Update Approved Plugins. Update approved third-party plugins when updates are announced at the bottom right of calibre's main window. Click the update message for calibre to download the updated plugin automatically and initiate the installation procedure. I like to update plugins as soon as possible to ensure there will be no age-related conflicts between older versions of plugins with newer versions of calibre.
  • Update Other Plugins. Update any other third-party plugins manually. Periodically check for updates at the relevant site, download updated plugins, and use Preferences/Plugins "Load plugin from file" button.

For Each Device:
  • Update Software. Where possible set these to automatically update on devices: firmware, operating system, reader apps, and other ebook-related apps. If automatic updating is not available, periodically look for updates and install the updates manually.

For Computer:
  • Update Software. Where possible set these to automatically update on computer: operating system, antivirus application, backup utility, ebook reader applications, office suites, word processors, editors, and other tools. If automatic updating is not available, periodically look for updates and install the updates manually. Generally keeping software up to date is easier and cheaper than dealing with performance, security, and maintenance problems from using old software.
  • Generate Backups. Set the backup application in the operating system to do automatic periodic backups that include all calibre libraries. I had to restore calibre libraries from backups several times in my early days with calibre when I made various blunders. My calibre libraries are backed up automatically to an external drive along with everything else on my internal drive. I don't want to rely solely on Dropbox or another cloud service for backups, because when I need to restore from backup those host servers or my internet connection might be down.
  • Use Antivirus Auto-Scan. Set the security/antivirus application to auto-scan all disk volumes but to exclude calibre Library folders from scans. When auto-scan is on, books are scanned for malware such as viruses at download time so there is no need to scan them again whenever calibre accesses book files. Excluding calibre library folders prevents the malware scan from causing slow-downs in performance whenever calibre accesses library files. If auto-scan was previously off, first have the security/antivirus application scan everything on all disk volumes (including any externals such as USB sticks, external drives, and reading devices that are not always connected)—this can be time consuming so I would start this to run overnight while the computer isn't needed for anything else. If any infected files are found during this initial scan, follow the instructions or Help given in the security/antivirus application to clean or delete the infected files.


Help
Spoiler:

Getting Started:
Additional Help:
  • Mouse Tips. Pay attention to Mouse Tips in calibre. These little boxes that come up when hovering the cursor over something contain important help messages about how calibre works. They are often more up to date than the user manual due to the rapid software update schedule.
  • calibre User Manual. Read various sections of the calibre User Manual. as necessary for initial learning about unfamiliar features. Use the manual as a reference.
  • MobileRead. Learn about ebooks, reading device(s), and other ebook related topics in the MobileRead Wiki. Read recent threads for discussion of various topics in the MobileRead Forums. The MobileRead forums accommodate a wide range of users from beginning to advanced in many ebook-related knowledge and skill areas.
  • calibre Forum. On MobileRead, browse the calibre Forum and its Sub-Forums for Library Management, Devices, Plugins, Conversion, Recipes, Catalogs, and Development. Search them to find information and discussion on topics of interest.
  • Stickies. Stickies are special threads at the top of a MobileRead forum. Read them carefully. Some of the stickies in the calibre forum and sub-forums contain more recent or detailed information than the manual or other help documentation.
  • Questions and Problems. Ask or report in the appropriate calibre forum when the calibre User Manual, stickies, and searching the forums don't seem to address a question or problem. Start a new thread. Put the specific question or problem in the title of the post with a question mark, and context and details in the body of the post. For technical problems, also include calibre version number (bottom left of calibre window) and operating system name and version.
  • Digital Rights Management. Digital Rights Management (DRM) refers to a variety of copy-protection schemes for ebooks and other media files. For background information on DRM for ebooks read the MobileRead Wiki article on DRM. Search the internet for "Apprentice Alf" and read that blog. DRM-removal tools or plugins are not supported or endorsed by MobileRead Forums or the calibre developers.
  • eBook Creation Tutorials. Read the tutorials about creating and converting ebooks. There are links to them from the official calibre Help page Getting Help with calibre.


Links
Spoiler:

Links Key:
  • Green. Internal links, for MobileRead Threads or Posts. For information, or in rare cases for a script and labeled "script". All are recommended.
  • Purple. External links, for wikis and other information sources. All are recommended.
  • Blue. External links, for software tools. Tools recommended by experienced people at MobileRead are indicated with "recommended". Any I haven't tried myself are indicated with "noted".

calibre:


General Topics:
Spoiler:

DRM (Digital Rights Management):
eBooks and Devices:
KISS Principle (Keep It Simple Stupid)
Metadata Standardization:
Regular Expressions (Regex or Regexp) and Renaming Files:
Workflow:


Formats and Conversions:
Spoiler:

Formats and Conversions, General Information:
CSS (Cascading Style Sheets):
EPUB (Electronic Publication):
Graphics:
HTML (HyperText Markup Language):
HTML—Alternate Browsers (HTML Readers):
MOBI (Mobipocket) and Amazon Kindle Formats:
PDF (Portable Document Format):
TXT, RTF, DOC, DOCX*, ODT*, PAGES* (Word Processing Formats —
Text, Rich Text Format, Document, Document XML-based, Open Document Text, Pages):


Version History:


Viewing all articles
Browse latest Browse all 31520

Trending Articles