More Improvements and Future Plans

It's time for an update. Since November 2nd, I've accomplished a few important developments in the slow evolution of Academist. In my last update, I gave myself a small to-do which I've now completed. I wanted to utilize the descriptions of Khan Academy topics in the search algorithm, rather than just the topic titles. To recap, I had a chemistry assignment that was about "colligative properties." This would be a great subject to review in Khan Academy, but Academist found nothing. I searched for "colligative properties" in Khan Academy, and sure enough, there was no page with that title; however, it was in a topic description. Integrating the topic descriptions in Academist was a matter of revising the code that downloads information from Khan Academy, and simply adding that information to the index of topics.

With a an even more "slightly-useful product," as that label still applies in this stage, I decided it would be a good time to add Academist to the Chrome Web Store. This proved not too difficult, and because it is not nearly ready for widespread use yet, I made it so only people with the link can access it, which I might as well put here.

I made a to-do list of the most basic tasks I had to complete on October 4th. I've completed the first to-do, which was developing the searching algorithm and automatically displaying the results under an assignment on School Loop. Now I have finally started on the second to-do: Creating spreadsheets of the table of contents of textbooks at Sage Creek. This is a very time consuming task, but I've already completed a good chunk of the table of contents of my AP Chemistry textbook, which is a relatively long one. Once this first textbook is fully integrated in Academist, I will encourage everyone in my chemistry class (who also uses Google Chrome) to download it, and I will finally start gathering feedback.

Speaking of which, I recently decided to to share the primitive version of Academist with a couple of my friends, only to find out that they both use Firefox. This means I'll definitely be looking into converting Academist to a Firefox Extension. It will be more work, but probably less so than convincing everyone who uses Firefox to switch to Chrome. Unfortunately, there’s not much I can do about those who use Safari and Internet Explorer, but to my knowledge, they are less popular.

Only time will tell if I can accomplish all I want to before the school year ends. That being said, here is a chart of what I plan to do in the coming months:

Small Adjustments for now, but Big Plans for the Future

Since two weeks ago, the constant flow of different assignments in School Loop has been a big help for me in improving Academist, but perhaps also a wake up call for the possible failures that Academist may have in the future.

The School Loop assignments that I have had have forced me to improve Academist's word filtering. Last post, I talked about how I made the Chrome Extension filter out common words in assignments like "study" and "quiz" that would interfere with the search algorithm. I made a list of similar words, including anything that came to mind. However, the words that came to mind were only a fraction of possible words and phrases that should be filtered out of an assignment name. I did not consider the words "lab" or "report" until I had a lab report due. Speaking of which, I also added the word "due" to the list. Here is an example of an assignment for which Academist would not have found Khan Academy topics if it weren't for improving the word filtering:

Academist filters out "Complete" and "Lab" in order to utilize the useful search term "Cons. of Momentum."

For the past two weeks, I have constantly been seeing assignments with words that were interfering with searching for Khan Academy topics. Adding these words has ultimately improved Academist, but the fact that I haven't stopped encountering them is a sign that I may never be able to make the word filtering robust. However, that is only one part of Academist, and certainly not the most important.

Testing Academist against all my assignments has led me to consider another important, yet simple, improvement. Khan Academy contains topics, but also descriptions of those topics. I also noticed that I had an assignment for chemistry class that involved "Colligative Properties," but there was no Khan Academy topic or video with that phrase. However, it was contained in the description of the Khan Academy topic, "Mixtures and solutions." This simply means that I will modify the code that generates the files used by Academist that store data from Khan Academy to include the description for each topic. Adding the description to be used in the search algorithm is also a simple process.

So far, I am still on to-do number one out of five on the to-do list I made on October 4th. There is still a lot to be done, so I will hope that I will have the time to make Academist into a truly useful tool.

A Slightly Useful Product

Last weekend, I was able to spend a lot of time on Academist. I completed the first thing on my to-do list from last post, which was to make a basic search algorithm to find Khan Academy topics and videos based on keywords that may appear in assignments.

The end result is that Academist will successfully display links to relevant Khan Academy pages for an assignment titled "Study for Quiz on Circular/Gravity," and it will know what type of Khan Academy resources they are (It says "Watch" if a video, "Study" if a broader topic, etc.). Luckily, I had this assignment on my list so that I could test Academist against a real assignment, instead of creating fake ones. Here is proof of it working:

Academist will automatically display the top five results using the search algorithm. This all came together thanks to a lot of intricate parts.

First, there was the database of Khan Academy topics. I stuck with only math and science to keep things simple, and I re-created the XML (discussed on 9/19) to include the type of resource for each Khan Academy page. I then had to parse that XML file in JavaScript and translate the data from the file into a JavaScript array.

The next part is the search algorithm. Of course, search algorithms are a widely useful tools that are used all over the place, so I wasn't going to reinvent the wheel. I just had to download a JavaScript library for a search algorithm, and I found one called "lunr." This library allowed me to input a query to search in a list, which in this case was list of all the math and science topics on Khan Academy. It was able to find similar, yet not exact matches, and also ignored common words such as "the." One thing that I did do myself was filter out common words that teachers post in assignments that would get in the way of searching for a topic, such as "study" and "quiz."

Those were the two most important parts, but I also had to learn how to format HTML elements, how to create a hyperlink with JavaScript, how to read local files in a Chrome Extension, and the list goes on. There are endless technical details that are involved in this project, but for the average non-computer-programmer, a picture of the code that makes Academist work might give you the gist of what I am working with:

Building the Chrome Extension - First Steps

After creating a foundation for accessing data for online resources, as discussed in my last post, I decided to start work on making the actual Chrome Extension. I started with the basics, using online tutorials to learn how to setup the file structure and add basic functionality.

One thing that Academist will have to do is be versatile in accessing information on the the School Loop Student Portal, namely assignments. I experimented with sifting through HTML elements and their contents using JavaScript, and I was able to make the program print out the name of an assignment– not too difficult, although it was slow getting used to the JavaScript programming language.

Chrome Extensions are also meant to alter the HTML of a webpage to add new UI features. Of course, Academist needs to do this. The most important feature of Academist will be buttons that are displayed under assignments in the Student Portal, which will open webpages of Khan Academy and possibly other resources related to an assignment. Enabling Academist to be able to take information from an assignment on School Loop and decipher what topic it is on is going to be a lot of work. That will happen later. For now, I'll just add a button that does nothing under every assignment. Currently, the Chrome Extensions displays a button with the text, "CLICK ME" under an assignment whenever the dropdown is activated, as shown below.

I could talk about some technical things now, like how I used a Mutation Observer in order to detect when the dropdown is first opened. Or I could talk about plans for the future, which I think will be more interesting.

Here's a reminder of the two main features I had planned for Academist: Find online resources to help understand topics on School Loop assignments based on keywords in those assignments, and find online resources to help understand topics on School Loop assignments based on textbook page numbers in those assignments and the topics of the sections that those page numbers are in. From my experience, math and science teachers will more often post textbook assignments with page numbers or section numbers rather than explicitly mention the topics that the homework is on. Because of this, I will focus more on the textbook aspect of Academist. But even with that feature, the topics in a textbook still need to be matched with the correct Khan Academy page, and any other online resource I choose to add into the mix.

This leaves some important to-do's:

• Create a basic word-matching algorithm and test it against fake assignments so that it can correctly determine corresponding topics in Khan Academy.
• Convert images of table of contents of textbooks at Sage Creek to useful spreadsheets.
• Convert the spreadsheets to a readable format for the Chrome Extension.
• Create an interface in Academist that prompts student to choose which textbooks they use for their classes.
• Put all of those things together in order to enable Academist to read page numbers in School Loop, then determine the topic name for that section in the textbook, then determine the Khan Academy page that explains that topic.

Starting a Database

The first step I saw fit to take for my project was making a database of topics and review videos for those topics. I started with one of the best homework review sites out there, so you are probably not surprised that I chose Khan Academy.

There is a big gap between the content on Khan Academy and a JavaScript program inside a Chrome extension, so I need to bridge that gap with a database that a JavaScript program can easily parse, AKA read and extract usable data from.

Luckily Khan Academy already has an organized database on their website (khanacademy.org/library), depicted below. Each subject brings you to a page with different topics, and within those topics are subtopics, and then finally are individual videos.

The next step is to create a list of every video in the Khan Academy library, organized into topics. Easy! Just start with individually copying the link to each of the 4,293 math videos...

Just kidding! That would be tedious beyond imagination. Instead I wrote a computer program to put all of the information into a file. I had many different options to choose for accomplishing this, but I decided to use the programming language I am most familiar with, Java (not the same as JavaScript), with a Java editor called Eclipse. Now skip this next sentence if you don't want to get any more technical: I used the Apache HTTP Client to connect to the webpages in the Khan Academy library, along with an HTML parser to extract the data from the website, such as the URL to a video or the title of a topic.

That process takes a while, simply because the program has to connect to the webpage for all 384 topics in the Khan Academy library, not to mention I have to figure out how to solve all the problems with the program before it works correctly (nothing ever goes right the first time with a computer program).

The end result is four XML files, one for each major subject on Khan Academy (math, science, humanities, and economics). XML is just a fancy, but also universally acknowledged data format that can be easily read in most programming languages. Here's what that looks like:

(this is a tiny fraction of the math topics XML file)

One unfortunate factor in the way I chose to work on this part of the project is the fact that the KhanAcademy website is constantly changing. I made this list initially a few months ago, but recently, Khan Academy added more content. That should have been easy to add, but they also changed the naming scheme of the HTML elements that define the topics and their respective URLs. Long story short: I had to rewrite half of the program. It would be crazy to think that won't happen again in the near future, but there's nothing I can do about that. However, there are many different directions this project can take me in, and this is only one small part of it.

What is "Academist?"

A wise old man once said: “Do. Or do not. There is no try.” So if you completely forgot from today's math lecture how to find the derivative of 9001x², you should not try it without knowing what you are doing. You should find a way to learn out how to do it, or it is not worth doing it at all.

I am working on a project that will encourage students to review specific topics on their homework so that they always understand the topic that they are learning in class.

It is a fact that students often complete their homework without fully understanding what they are doing. They merely try, but that is not an option worth taking. This is very evident with math. Students turn their homework into guesswork and come to school the next day with a million questions for the teacher. But it does not have to be this way.

Of course, teachers are there to help, but that does not mean there isn’t value in teaching yourself. There are incredible resources on the internet that make topics more understandable, but students don’t take advantage of them nearly as much as they should. I am going to change that at my high school, Sage Creek, and potentially at other schools as well.

I am going to do it with a single piece of software, which I am calling "Academist." It has one primary function, which is to provide you with links to internet resources to help you with your homework.

It will automatically tailor to your needs by using keywords that appear in your School Loop page, such as “derivatives,” and search for learning tools that explain that topic. For example, Khan Academy.

There are also some great YouTube channels for reviewing class topics, such as Bozeman Science, which has an abundance of videos explaining, chemistry, physics, and biology topics.

I am planning to focus on videos for learning resources, because I believe they are the easiest way to learn and understand a topic, since they combine both audio and visual learning. I will still want to get feedback from students about what kinds of resources they like to use.

The hope is that by providing these resources in an automated way, students will be more likely to access them if they don’t understand their homework.

The software will be a Google Chrome extension, which will allow it to automatically alter what you see on your School Loop page when you visit it on your computer. This has many advantages over a standalone application. Students don’t have to change how they access School Loop, and I don’t have to reinvent the wheel by creating a brand new user interface for a service that already has a relatively good one. This will allow me to focus on the more important, yet hidden aspects of the project, which include developing efficient search algorithms to find relevant online learning resources based on the words that appear in School Loop.

I see this app working mainly with math and science, because the their curriculums are relatively formulaic. But history also has potential, since students may want to review certain chapters without just reading through them twice in the textbook.

Of course, no software is complete without a logo. This is currently what I am using, simply a combination of a couple clip art images:

The reason I have decided to take on this project is because I have a lot of experience with developing software, and this would be a chance for me to develop software that is truly useful and will have an impact on my peers.

So far, I have done a fairly limited amount in developing Academist. What I have done is created a formatted list of every topic in Khan Academy that can be easily used by a computer program. I've also begun programming the Chrome extension to add a button under every assignment in School Loop.

I have mentor, Robert Nance, a software engineer at Qualcomm, who will be giving me advice as I develop this project. He teaches me in computer programming every week and has helped me a lot in my programming projects in the past.

The biggest obstacle for Academist is the potential for keywords in School Loop to be few and far between. For example, a test on derivatives may be titled by a teacher simply “Chapter 1 test” in School Loop, and homework assignments are often posted in the form of textbook page numbers. However I can use page numbers and chapter numbers to my advantage.

For all the textbooks at Sage Creek, I can convert the table of contents into a readable format for Academist, and Academist can then match a page number with a topic using the table of contents, and finally look for that topic in online learning resources. I have made some progress in making this happen by taking pictures of the table of contents of nearly all the math, science, and history textbooks at Sage Creek. I will convert the table of contents into a readable format for Academist using a combination of OCR and tedious spreadsheet writing.

Because of its useful tools, I believe Academist has the potential to be very integrated in the academic lives Sage Creek students, and the hope is that it will increase the degree to which students self-teach to review tough topics. In any case it will greatly increase the efficiency by which students can get homework help.

Below is a video about Academist that I used as a visual aid when I pitched it last June at my Sage Creek. The video lacks the verbal presentation that I delivered during the pitch, but it may add some depth to this blog post, as it has some visuals of what the Academist will actually look like.