Epub summerizer
I made a rather silly thing this week-end. I finally vibecoded an idea I had for a long time: a program that takes an EPUB for input and outputs an executive summary from it, chapter by chapter, with a basic character map, and a context coming from a local Kiwix server hosting a copy of Wikipedia ; and the brain behind it is a local LLM hosted on one of my machines. That's a lot of moving parts actually.
With a few iterations, I got it working for fiction and non fiction, with EPUB having various degrees of quality.
The silly part is that, I find reading more enjoyable now that my books are pre-mapped. I always found reading a book exhausting because of the so many details I would (should) write down and at the end, most of the time, I would hardly remember anything of it.
I spent my whole weekend moving back and forth between the summaries and the original material.
I am indeed a little ashamed of my laziness and own inability to do something as simple as reading a book, but as it is enjoyable that way and does not cost anything to anyone other that my computer's own inference cost ; I consider it a success.
Here is an example of what it produced, it took around 20 minutes:
================================================================
FICHE DE LECTURE — Contre le bourrage de crâne (Albert Londres)
================================================================
1. AUTHOR & CONTEXT
----------------------------------------
(Source: local Wikipedia article "Albert Londres")
Albert Londres was a French journalist and writer who lived from 1 November
1884 to 16 May 1932. He was one of the inventors of investigative journalism,
known for not only reporting news but also creating it from a personal
perspective. During the First World War, Londres served as a war
correspondent, traveling across multiple fronts, including Belgium and Italy,
and documenting the horrors of the conflict. His book "Contre le bourrage de
crâne" is a collection of his newspaper articles, written in a sharp, cynical
style that reflects his experiences and critiques of the war. The work
highlights the inhumanity of the conflict, particularly the actions of the
Germans, and was published during the war, offering a firsthand account of the
war's brutal realities. After the war, Londres interviewed the German
population, who remained unaware of their defeat.
2. SYNOPSIS
----------------------------------------
"Contre le bourrage de crâne" by Albert Londres is a collection of reflections
and reports from the trenches of World War I, capturing the experiences of
French soldiers and the broader context of the war. The book explores themes
of resistance against military censorship and propaganda, as well as the harsh
realities of combat, the psychological toll on soldiers, and the symbolic
significance of the French flag and national identity. Londres's work is
marked by his defiance of state control, as seen in his reporting and the
consequences he faces, including being marked as 'indésirable' and placed on a
blacklist. The narrative includes vivid descriptions of battles such as the
Battle of the Somme, the Battle of Flanders, and the capture of key positions
like 304, emphasizing the brutal and industrial nature of modern warfare.
Through these accounts, Londres conveys the transformation of soldiers into
disciplined, practical individuals, contrasting their romanticized heroism
with their grounded, realistic understanding of war. The book also touches on
the emotional and psychological impact of war, the shared suffering among
troops, and the symbolic representation of French identity in the region of
Alsace. The book is structured around various sections that provide a
multifaceted view of the war, blending personal accounts with broader
historical and strategic observations. Each section, such as "De l’armée
Sarrail à l’armée Pétain" and "La cote du Talou après la victoire,"
contributes to the overarching themes of war's devastation, the resilience of
soldiers, and the symbolic importance of national identity. Londres's writing
reflects on the psychological and emotional toll of combat, as well as the
strategic decisions that shaped the war front. The inclusion of specific
battles, such as the Battle of the Somme and the Battle of Flanders, allows
for a detailed examination of the war's impact on both the soldiers and the
civilian population. The book also features independent stories, such as the
tale of Eitel, a thief with a deep connection to châteaux, and the narrative
of a young woman in Massevaux who finds solace in the community's support
after her fiancé's death. These individual pieces, while distinct, are united
by their exploration of war's human cost and the enduring presence of French
identity in the face of adversity.
3. KEY ENTITIES
----------------------------------------
Concepts & Themes:
- French Soldiers — Represent the central theme of the book, highlighting
their resilience, transformation from romanticized heroes to practical
individuals, and the psychological toll of war. They are portrayed as
enduring the harsh realities of combat and the futility of war.
- French Flag — A powerful symbol of French national identity and resilience
throughout the book. It is referenced in multiple sections, representing
the soldiers' connection to their country and the enduring presence of
French identity in the face of war.
- War's Futility — A central theme of the book, emphasizing the senseless
nature of war and the loss of human dignity. It is explored through the
soldiers' experiences, the destruction of cultural symbols, and the tragic
fate of young soldiers.
- French offensive in Flanders — A major military operation by the French
army in October 1917, which included the bombing of Essen by Sergeant Luc
Jardin. It was a significant event that demonstrated the French military's
capabilities and the challenges faced by the German forces.
Places:
- Battle of the Somme — A major World War I battle that serves as a backdrop
for the book's exploration of war's devastation and the soldiers'
experiences. It is referenced in the context of the soldiers' physical and
psychological exhaustion.
- Battle of Flanders — A significant World War I battle that is central to
the book's narrative, symbolizing the ongoing struggle and the impact of
war on both the battlefield and the French identity. It is referenced in
multiple sections, emphasizing its strategic and symbolic importance.
Organizations:
- French Army — The central force in the book, representing the collective
experience and sacrifices of French soldiers during World War I. It is
depicted as enduring the brutal realities of war and the challenges posed
by the environment and enemy forces.
- German Forces — The opposing force in the book, symbolizing the harsh
realities of war and the challenges faced by French soldiers. They are
portrayed as a formidable and relentless enemy, contributing to the book's
themes of war's futility and the loss of human dignity.
4. SECTION SUMMARIES
----------------------------------------
Section 1 — 003.html
Albert Londres resists military censorship and propaganda during World War I,
often subverting it through his reporting. He faces severe consequences,
including being marked as 'indésirable' and placed on a blacklist by the state
major. His biographer notes his defiance, and while the military files a
complaint against him for insolence and insubordination, the editor of the
Petit Journal supports his work.
Section 2 — De l’armée Sarrail à l’armée Pétain
Londres reflects on the devastation of France during World War I, comparing it
to Pompeii, and discusses the transformation of soldiers into practical,
experienced individuals. He highlights their resilience and the shift from
romanticized heroism to a more grounded, realistic understanding of war. The
passage also reflects on the hardships faced by French soldiers in the Orient
during World War I, emphasizing their endurance and the grim reality of war.
Londres contrasts the romanticized heroism of soldiers with their practical,
experienced nature, highlighting the shared suffering and silence of the
troops. The passage underscores the intense conditions and the soldiers'
unyielding spirit, reinforcing the theme of resistance against military
censorship and propaganda.
Section 3 — Une forge de la victoire
The passage 'Une forge de la victoire' from 'Contre le bourrage de crâne' by
Albert Londres depicts the transformation of French soldiers into disciplined,
practical individuals during World War I. It emphasizes the shift from
romanticized heroism to a grounded understanding of war, highlighting the
harsh realities of combat such as the use of machine guns and the systematic
nature of attacks. The battlefield is compared to an industrial setting,
reflecting the overwhelming power of modern warfare. The text also reflects on
the loss of traditional soldier roles and the psychological toll of war, as
seen through the character of Gouraud. The intense and chaotic nature of
combat is portrayed, emphasizing the futility of war and the loss of human
dignity.
Section 4 — La relève au sortir des flammes
The passage 'La relève au sortir des flammes' from 'Contre le bourrage de
crâne' by Albert Londres describes the aftermath of the Battle of the Somme,
focusing on the physical and psychological toll on French soldiers. It
highlights the transition from the brutal reality of war to the necessity of
replacement, emphasizing the soldiers' exhaustion and the grim conditions they
face as they emerge from the trenches.
Section 5 — Eitel, prince de Prusse, voleur, etc.
The passage introduces Eitel, prince de Prusse, as a thief with other
qualities, highlighting his ability to recognize and interact with objects. It
reveals his habit of moving between châteaux, including Frétoy and Avricourt,
and his emotional attachment to them. The text also mentions his tact and
social connections, as well as the consequences of his actions, including a
complaint from the owner of Frétoy.
Section 6 — Devant Craonne C’est plus fort que Verdun
The passage describes the intense battle at Craonne in July 1917, comparing it
to Verdun in terms of ferocity and scale. It highlights the overwhelming
artillery and the psychological impact on soldiers, noting that the
Tourangeaux were the ones who suffered. The text also mentions the strategic
decisions of the kronprinz, who chose the stosstruppen for the attack,
emphasizing that Craonne was more than Verdun, but also less, as Pétain
reviewed the front elsewhere.
[...] and it continues, section after section.
As you can see it's far from perfect but for a personal usage given the constraint of a local LLM 8B model running on a 16Gb Nvidia Jetson, that's "good enough". Interestingly, the original material for that particular example was in French. The real danger is, like anything generated by an LLM, that it "looks" plausible ; so it won't replace reading the original book, obviously.
To obtain such results, the program executes the following steps:
- parse the arguments passed when starting the program with argparse. Info like the LLM or kiwix provider's IP:port and input/output files are passed as arguments.
- initialize a "memory" JSON that stores the temporary working memory used by the program. That JSON contains the summaries of each chapters, the overall context, the entities (characters, places) and ideas of the "chunk" it's currently processing.
- parse the EPUB, extracts the metadata, and the overall structure of the content
- connects to the LLM via the open-ai api, the LLM is served by llama.cpp
- get the context around the author and book from a kiwix-serve hosting a offline version of Wikipedia (this part is a bit flickery when the model retrieves the wrong article)
- loops on the book's sections. I don't call it chapter because some poorly formatted epubs are just throwing a wall of text with hardly no pre-made split. For each section, the program calls the LLM with the raw passage, a "frame" of the previous sections and the summary of the section before. This step feeds the structured JSON. It also extracts the "entities" (place, characters) based on the generated summary
- generate the synopsis
- generate the whole report in the form of a text file.
During the different iterations, the model took the liberty to implement a couple of features I did not specify:
- a resume function. Basically, in case of crash, the program can recover the JSON and continue from the last completed session, no need to restart the whole book
- a fancy progress bar to measure the advancement and estimate the time time remaining
- a `--verbose` argument to display the debug logs
As carving the steps to achieve the desired result was within my skill-set, coding each and every routine would have been out of reach for me without the help of a AI-assisted coding.
Don't worry, I won't pollute any public GitHub with my slop. ^.^