diff --git a/web/README_WGU.org b/web/README_WGU.org index ee27414..bd6e3bf 100644 --- a/web/README_WGU.org +++ b/web/README_WGU.org @@ -145,8 +145,6 @@ I've also been writing limericks my entire life and hold the International Limer :CUSTOM_ID: executive-summary :END: -Write an executive summary directed to IT professionals that addresses each of the following requirements: - ** Decision Support Opportunity Songwriters expend a lot of time and effort finding the perfect rhyming word or phrase. RhymeStorm™ is going to amplify user's creative abilities by searching its machine learning model for sensible and proven-successful words and phrases that meet the rhyme scheme and meter requirements requested by the user. @@ -225,8 +223,6 @@ In anticipation of user growth, I'll be deploying the final product on DigitalOc ** Requirements Validation And Verification -the methods for validating and verifying that the developed data product meets the requirements and subsequently the needs of the customers - For the known requirements, I'll perform personally perform manual tests and quality assurance. This is a small enough project that one individual can thoroughly test all of the primary requirements. Since the project is broken down into isolated sub-projects, unit tests will be added to the sub-projects to make sure they meet their own goals and performance standards. @@ -235,8 +231,6 @@ The final website will integrate multiple technologies and the integrations won' ** Programming Environments And Costs -the programming environments and any related costs, as well as the human resources that are necessary to execute each phase in the development of the data product - One of the benefits of a Hidden Markov Model is its relative computational affordability when compared to other machine learning techniques, like Deep Neural Networks. We don't require a GPU or long training times on powerful computers. The over 200,000 songs obtained from http://darklyrics.com can be trained into a 4-gram Hidden Markov Model in just a few hours on a consumer laptop. @@ -829,7 +823,7 @@ You'll see 3 input fields. The first input field is for a word or phrase for which you wish to find a rhyme. Submitting that field will return three visualizations to help you pick a rhyme. -The first visualization is a scatter plot of rhyming words with the "quality" of the rhyme on the Y axis and the number of times that rhyming word/phrase occurrs in the training corpus on the X axis. +The first visualization is a scatter plot of rhyming words with the "quality" of the rhyme on the Y axis and the number of times that rhyming word/phrase occurs in the training corpus on the X axis. [[file:images/wgu-vis.png]] @@ -846,11 +840,15 @@ The third visualization is a table that lists all of the rhymes, their pronuncia :CUSTOM_ID: remaining-documentation :END: -Create each of the following forms of documentation for the product you have developed: - ** Business Vision -Provide rhyming lyric suggestions optionally constrained by syllable count. +Supercharge songwriter's abilities with automated rhyming lyric suggestions for brainstorming. + +Without the physical constraints imposed by paperpack rhyming dictionaries, and with the full power of machine learning training, RhymeStorm™ will find rhymes don't show up in typical rhyming dictionaries. + +Rhymes and lyric suggestions will further be honed to target specific genres based on the training data set. + +These two features combine with the speed of modern-day processing to provide rapid-fire rhyming suggestions never before seen. *** Requirements @@ -929,10 +927,18 @@ and more. ** Visualizations +RhymeStorm™ provides three visualizations to help songwriter's find the perfect lyric. + +The first visualization is a scatterplot comparing rhyme quality to frequency that the rhyming word or phrase appears in the training corpus. + [[file:images/rhyme-scatterplot.png]] +The second visualization is a word cloud where each word's size is in proportion to the frequency with which the word appears in the training corpus. + [[file:images/wordcloud.png]] +And the third visualization is a sorted table of rhyme suggestions. The rhymes are sorted first by quality and then by popularity. + [[file:images/rhyme-table.png]] ** Accuracy @@ -1035,6 +1041,8 @@ Here is an example of the test suite for the code related to syllabification: [[ ** Source Code +I wrote three Clojure libraries and one Clojure application that combine to make RhymeStorm™. + *** Tightly Packed Trie This is the data structure that backs the Hidden Markov Model. diff --git a/web/resources/public/README_WGU.htm b/web/resources/public/README_WGU.htm index 4a35f20..53ee692 100644 --- a/web/resources/public/README_WGU.htm +++ b/web/resources/public/README_WGU.htm @@ -3,7 +3,7 @@ "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
- +Hello! I hope you enjoy your time with this evaluation! @@ -362,20 +362,20 @@ After I describe the steps to initialize a development environment, you’ll
It’s probably not necessary for you to replicate my development environment in order to evaluate this project. You can access the deployed application at https://darklimericks.com/wgu and the libraries and supporting code that I wrote for this project at https://github.com/eihli/clj-tightly-packed-trie, https://github.com/eihli/syllabify, and https://github.com/eihli/prhyme. The web server and web application is not hosted publicly but you will find it uploaded with my submission as a .tar archive.
./db/run.sh && ./kv/run.sh to start the docker containers for the database and key-value store.
@@ -400,12 +400,12 @@ It’s probably not necessary for you to replicate my development environmen
./db/run.sh && ./kv/run.sh to start the docker containers for the database and key-value store.
@@ -436,8 +436,8 @@ It’s probably not necessary for you to replicate my development environmen
Songwriters, artists, and record labels can save time and discover better lyrics with the help of a machine learning tool that supports their creative endeavours. @@ -449,8 +449,8 @@ Songwriters have several old-fashioned tools at their disposal including diction
How many sensible phrases can you think of that rhyme with “war on poverty”? What if I say that there’s a restriction to only come up with phrases that are exactly 14 syllables? That’s a common restriction when a songwriter is trying to match the meter of a previous line. What if I add another restriction that there must be primary stress at certain spots in that 14 syllable phrase? @@ -466,8 +466,8 @@ And this is a process that is perfect for machine learning. Machine learning can
RhymeStorm™ is a tool to help songwriters brainstorm. It provides lyrics automatically generated based on training data from existing songs while adhering to restrictions based on rhyme scheme, meter, genre, and more. @@ -495,8 +495,8 @@ This auto-complete functionality will be similar to the auto-complete that is co
The initial model will be trained on the lyrics from http://darklyrics.com. This is a publicly available data set with minimal meta-data. Record labels will have more valuable datasets that will include meta-data along with lyrics, such as the date the song was popular, the number of radio plays of the song, the profit of the song/artist, etc… @@ -508,8 +508,8 @@ The software can be augmented with additional algorithms to account for the type
This software will accomplish its primary objective if it makes its way into the daily toolkit of a handful of singers/songwriters. @@ -529,8 +529,8 @@ Another example is the package that turns phrases into phones (symbols of pronun
This project will be developed with an iterative Agile methodology. Since a large part of data science and machine learning is exploration, this project will benefit from ongoing exploration in tandem with development. @@ -546,8 +546,8 @@ The prices quoted below are for an initial minimum-viable-product that will serv
Funding requirements are minimal. The initial dataset is public and freely available. On a typical consumer laptop, Hidden Markov Models can be trained on fairly large datasets in short time and the training doesn’t require the use of expensive hardware like the GPUs used to train Deep Neural Networks. @@ -631,17 +631,17 @@ These are my estimates for the time and cost of different aspects of initial dev
Web scraping, the method used to obtain the initial dataset from http://darklyrics.com, is protected given the ruling in https://en.wikipedia.org/wiki/HiQ_Labs_v._LinkedIn (HiQ Labs v. LinkedIn 2021). @@ -653,8 +653,8 @@ The use of publicly available data in generative works is less clear. But Micros
I have 10 years experience as a programmer and have worked extensively on both frontend technologies like HTML/JavaScript, backend technologies like Django, and building libraries/packages/frameworks. @@ -670,13 +670,10 @@ I’ve also been writing limericks my entire life and hold the International
-Write an executive summary directed to IT professionals that addresses each of the following requirements: -
Songwriters expend a lot of time and effort finding the perfect rhyming word or phrase. RhymeStorm™ is going to amplify user’s creative abilities by searching its machine learning model for sensible and proven-successful words and phrases that meet the rhyme scheme and meter requirements requested by the user. @@ -688,8 +685,8 @@ When a songwriter needs to find likely phrases that rhyme with “war on pov
Songwriters spend money on dictionaries, compilations of slang, thesauruses, and phrase dictionaries. They spend their time daydreaming, brainstorming, contemplating, and mixing and matching the knowledge they acquire through these traditional means. @@ -709,8 +706,8 @@ Computers can process and sort this information and sort the results by quality
We’re all familiar with dictionaries, thesauruses, and their shortcomings. @@ -726,8 +723,8 @@ RhymeZone is limited in its capability. It doesn’t do well finding rhymes
The initial dataset will be gathered by downloading lyrics from http://darklyrics.com and future models can be generated by downloading lyrics from other websites. Alternatively, data can be provided by record labels and combined with meta-data that the record label may have, such as how many radio plays each song gets and how much profit they make from each song. @@ -751,8 +748,8 @@ Each new model can be uploaded to the web server and users can select which mode
RhymeStorm™ development will proceed with an iterative Agile methodology. It will be composed of several independent modules that can be worked on independently, in parallel, and iteratively. @@ -776,8 +773,8 @@ Much of data science is exploratory and taking an iterative Agile approach can t
I’ll start by writing and releasing the supporting libraries and packages: Tries, Syllabification/Phonetics, Rhyming. @@ -832,13 +829,9 @@ In anticipation of user growth, I’ll be deploying the final product on Dig
-the methods for validating and verifying that the developed data product meets the requirements and subsequently the needs of the customers -
-For the known requirements, I’ll perform personally perform manual tests and quality assurance. This is a small enough project that one individual can thoroughly test all of the primary requirements.
@@ -853,13 +846,9 @@ The final website will integrate multiple technologies and the integrations won&-the programming environments and any related costs, as well as the human resources that are necessary to execute each phase in the development of the data product -
-One of the benefits of a Hidden Markov Model is its relative computational affordability when compared to other machine learning techniques, like Deep Neural Networks.
@@ -878,8 +867,8 @@ All code was written and all models were trained on a Lenovo T15G with an Intel