Photo by Emil Widlund on Unsplash

Introduction

When I first stumbled across Anne Bogel’s website and bookish empire, I loved the idea that she has created a “reading life”. Over the last few years, I’ve gently grown my own life by adding in things like:

• diversifying and decolonizing my bookshelves
• drawing inspiration to broaden my reading life through a wide array of reading challenges
• embracing my library
• enjoying audio books more
• sharing as many book recommendations and reviews with people as I can

I’m beginning to think about my 2024 reading life goals. One thing is clear, I want to add some structure to my reading life. Top of mind for me are three things:

1. Developing a content rating system/framework that accounts for the wide array of content I consume
2. Developing (and using) a set of core content - things that constantly bring me joy and help me embrace growth and healing
3. Using a digital reading journal/notebook

In this article, I want to focus on that first item - building a content rating system or framework for the wide array of content I consume.

Why a rating system or framework?

I consume a LOT of content - from blog posts to physical books to book chapters to web content to podcasts to audio books to documentaries to online courses. Learning is a hallmark of who I am and how I interact with the world. And I love to share things that make me think of the people in my life with them. I find it particularly endearing when they share back that a recommendation was valuable to them and when they share recommendations with me.

When I partake of content people recommend to me, I want to share my honest feelings about that content with them.

What I’ve found over time is that I struggle to recall salient points and details of content after I’ve consumed it. Digital note taking in tools like Obsidian has helped immensely, as has my use of Readwise. However, I feel that a more structured rating system or framework would further aid in helping me recall what I liked and didn’t like about a particular piece of content and more carefully articulate my own feelings rather than being swayed by the views of others. In short, I think it will help me to develop stronger convictions about the content that I consume, while also actively challenging those convictions with new information to better improve my learning.

Criteria for a rating framework

A rating framework that can be applied to such a wide array of content needs some strong consideration in its development. I also imagine that it is going to take some iteration to get it right so I plan to ensure that, over time, I come back and evaluate my framework to see how well it is serving me. If you choose to test my framework, or if you have a framework of your own, I hope that you’ll offer feedback as well.

With that said, here are my initial thoughts on my needs specific to a rating framework. I think it’s important to figure these out before I actually settle on the rating framework criteria and process itself.

1. It must be easy and efficient to apply. I’d like to think about the framework only as much as it is necessary to quickly apply it to a specific piece of content.
2. If I’m going to rate a piece of content, then I must first write/share something about that piece of content in my own words. So, that may mean something as simple as the blurb I put at the end of my newsletter about different pieces of content I consume, thoughts on a specific book, such as my thoughts on Prince Harry’s book Spare or my monthly reading life roundups. I’ve also noted that I’m actively trying to expand my Reading Life to my “Content Life” which is a change I’ll continue to explore and may write about or publish about in other ways in the future.

This specific item is important because I consume a lot of content that isn’t, at least in the moment, specifically worth sharing in any meaningful way. Until I’m ready to synthesize that content into my own words, it’s likely not worth taking the time to rate and review. This also adds to the ease and efficiency of the overall system. I’m not seeking to rate every piece of content I consume. I’m seeking to think critically about what content I might share with others and why I’m sharing it.

1. Finally, the framework needs to be flexible. What applies to fiction doesn’t always apply to non-fiction and vice versa. Spoken and performed content is different from text content. I want one framework I can apply to all types of content.

Inspiration from Existing Rating Systems

For this specific section of the article, I asked ChatGPT to help me identify examples of content rating systems. Unfortunately, ChatGPT was having a bad day and when I asked it to provide multiple examples of content rating systems that I could use as inspiration and to feed my own thought processes, it wouldn’t return results. So, personal web searching it is.

Let’s start with one of the reasons I started down this path. I was inspired by Jashii Corrin to come up with and elaborate on my own content rating system. Jashii is a bullet journaler who now develops content on YouTube full time. The best explanation I’ve found of her book rating system is in this Livestream she did about her reading journal. Jashii uses a different system for fiction vs non-fiction systems. I want to use the same system but want it to be flexible enough to adapt to the different types of content and media. Her system is based on one by G at Bookroast. (Bookriot did a written review of the CAWPILE rating system that G developed if you’d like to read about it.) Jashii uses a 5 point scale for each of the various criteria rather than a 10 point scale. As I’m developing my framework, I’m considering a weighted system that is then averaged by the total of number points available to that book. Some of that decision may come down to how challenging it is to develop the spreadsheet to track ratings.

On Book Street is another site I’ve frequented many times. The criteria that are suggested in the book rating guide on that site are:
- initial response
- will you recommend?
- will you re-read?
- style
- plot/structure
- character
I really like these criteria and I believe most of them will find a place in my final rating system. However, what I don’t like about that rating system is that there is no direct connection between how a book scores on each of these criteria and the actual star rating. I am a quantitative person and would like to be able to look back a year from now and see why I rated something high or low and see if my opinion on that specific aspect has changed over time. This is one of the areas where my conversation with ChatGPT was valuable in that it suggested building in a feedback loop to my rating system. I can easily see revisiting or even scheduling specific reviews of my ratings so that I can see how that read or knowledge has stuck with me.

As to character, I think the other element of this that could weave into non-fiction is story-telling. All content producers are telling some kind of story. Did they do it well or did they do it badly?

Megan at Reading Books Like A Boss explains her star rating system but doesn’t have specific criteria. That said, I do appreciate the point that Megan will only rate books (or in my case content) that she has finished. I agree with her that it is unfair to rate a piece of content you don’t finish. I also appreciate that a 3 star (out of 5) rating is given to content she considers “good”. I tend to be an “easy” grader so another potential benefit of developing a more structured rating system is getting to a more honest evaluation of the book based on its merits in a number of areas and not one overall star rating.

Sarah at Regal Reads has some specific criteria in her list that I really like and definitely want to consider. From her list, I’d add to my potential criteria:

• pacing/appropriateness of length
• world-building
  World building, particularly in fiction work, is incredibly important to me. I want the world to be consistent and to make me want to know more about it. For non-fiction work, I think this is analogous to a word that Sarah applies to fiction, but I think applies better to non-fiction and that is coherence. The definitions of coherence are:
• the quality of being logical and consistent.
• the quality of forming a unified whole.
  So, I can easily see using world-building and coherence interchangeably between fiction works and non-fiction works.

Olivia at the Candid Cover describes only her star rating system but something I took from that is excitement to read/ease of finishing. There are many books I’ve simply abandoned (which would go on my Did Not Finish/DNF list) for one reason or another but there are other books that I’ve finished but done so as a really slow burn over time which is not my norm.

I was particularly impressed by Lexi at Just Peachy Editorial. Lexi uses the acronym SPECS to describe her book rating system. Each letter has a few “sub criteria” that might fall into it, but I like the ease of remembering the system with a simple acronym. Another good acronym based rating system is from Fiona at Fi’s Bibliofiles. Her system is the ESCAPE system. Ellie at Read to Ramble developed her system from these as well as the CAWPILE system mentioned above and developed ASPECTS. I think it’s clear, I’d like my system to have a meaningful, at least to me, acronym or pneumonic attached to it.

I appreciated some of the things that Rose at Rosetree Energy Spirituality pointed out about non-fiction books. Among these are:

• originality
• can I apply this tomorrow? - Rose calls this high-concept versus generous but it is an important issue in books today. Was there enough information in the book for me to actually act on or do I have to buy 10 other things by the content producer to actually get the depth of information
• truthfulness/validity to me - I think, like coherence, this could fit in as a substitute for world-building in the fiction world in so much as it applies to how did it apply to my world and my world view.
  With that roundup of various rating systems gathered, I felt I had a set of criteria I could start with to develop my system. Further search could have revealed additional categories to consider. However, keeping in mind that I tend to overcomplicate things and I want this to be easy and efficient, it was better to stop here and refine the ideas I’ve already gathered and then do an assessment to see whether anything was missing or felt out of balance.

Initial Criteria Set

So, from all of those different systems, I have the following initial criteria list:

• initial response
• will you recommend?
• will you re-read?
• style
• plot/structure
• character/story telling
• pacing/appropriateness of length
• world-building/coherence
• excitement to read/ease of finishing/pacing
• originality
• can I apply this tomorrow
• truthfulness/validity to me
  I also want a memorable acronym that I can use for the system.
  I will only rate content I’ve finished.
  I will set 3 (on a 5 point scale) as good. To get a 5 means it is one of the best pieces of content I’ve ever consumed.

Refining the Criteria

This initial criteria set is too long to fit with my desire for the framework to be easy and efficient to use. The criteria also have some overlaps in them. I started by grouping the criterion into logical groups and giving each one a definition

• Initial Response - When I finish a book, I’m often left wanting more or to run right out and tell everyone about what I learned from the book. I also decide if I might want to re-read a book or not. Whether or not I do is likely a good thing to decide as I review ratings over time in the feedback loop.
• Structure, Pace, and Readability - Structure takes on how the book was set up, how well it flows, whether the length is appropriate, and the overall readability of the book.
• Coherence/World Building - In fiction, the sense of place the author creates or the entire world they create is extremely important to the overall quality of the book. In non-fiction it's how factual this book feels and how valuable and easy to apply in everyday life the information is.
• Storytelling - How the author conveys their message
• Character(s) - A book has characters and it has character (the Merriam Webster definition is “the mental and moral qualities distinctive to an individual” which I think are also appropriate to say as “the mental and moral qualities distinctive to a book”. For fiction books, both definitions could be taken into account as in the characters in the book as well as the character of the book. For non-fiction books this is primarily just the character of the book. Do I feel like the author honored the intended audience and met the goals they stated for the book?
• Creativity/Originality - I love to read original ideas and imaginative worlds.

I really wanted no more than 5 criteria but I feel like each of these is important. Over time, perhaps I can get it down to 5 items, but for now, I think I’ll go with these 6. I also failed to come up with an easy to memorize acronym, but if I rearrange them by letter, I get an I word, 2 S words, and 3 C words. For now, that’s easy enough to remember. So, here are my final criteria in order.

• Initial response
• Structure
• Storytelling
• Coherence/World Building
• Character(s)
• Creativity

Using the Criteria

Now that the criteria are established, I’m left with the practical question of how I will use them.

I initially considered creating a weighted scale. However, as I look at my list of criteria, I do think they are all of equal importance to the overall quality of a rating. Here is the list of criteria again:

• Initial response
• Structure
• Storytelling
• Coherence/World Building
• Character(s)
• Creativity

I’m going to use a 5 point scale where:

• 5 means exceeds all expectations/among the very best I’ve ever consumed
• 4 excellent in many ways
• 3 good. Quality content, I would not hesitate to recommend to the right person.
• 2 fair, but lacking in some areas
• 1 poor
• 0 among the worst I’ve ever read
  I don’t believe I will give many 0’s and 1’s as I’m not sure I will finish consuming content that I would rate a 0 and 1. That said, I wanted to leave it available to me as a book could score a 3 in one area with a 0 or 1 in another area.

Each criteria will receive a score between 0 and 5. I’ll then total the score for a piece of content and average those scores. This means if I give a book the following scores:

• Initial response - 3
• Structure - 2
• Storytelling - 3
• Coherence/World Building - 2
• Character(s) - 2
• Creativity - 2
  the overall rating for the book would be (3+2+3+2+2+2)/6 = 14/6 = 2.33

The final decision is how this translates into star ratings. In my own personal system, I can easily use a 2 decimal format like above, using standard rounding rules. In a standard 5 star format, each rating will have to be rounded to the nearest whole number so this 2.33 would become a 2 star rating.

I’m looking forward to giving this new system a try in my next reading life roundup which will come out at the beginning of December and then to use it for other content reviews in 2024. I feel like this will greatly enhance the consistency of my ratings.

Do you have a standard way you rate content or do you just go with the flow and your mood at the moment?

Tracking the Data

I’ve developed a simple Google Sheet that I’m testing for capturing all of the data. I also developed an Obsidian template that I’m testing for capturing my reviews. I’ve previously haphazardly written reviews between Ulysses and Obsidian. I may end up going the other way (Ulysses to Obsidian), but I prefer starting everything in Obsidian when possible. The specific workflow will have to be developed as I move forward from basic testing to actual implementation. I am sure I will refine the workflow.

After I feel like the process is working well, hopefully early in 2024, I’ll share the template and spreadsheet with you. For now, here is a quick peak at the spreadsheet: