<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Data Ninja</title>
	<atom:link href="http://blog.zarang.com/feed/" rel="self" type="application/rss+xml" />
	<link>http://blog.zarang.com</link>
	<description>Finding the data scientist within</description>
	<lastBuildDate>Mon, 16 Jul 2012 01:38:42 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.4</generator>
		<item>
		<title>Startling word discovery</title>
		<link>http://blog.zarang.com/2012/05/25/startling-word-discovery/</link>
		<comments>http://blog.zarang.com/2012/05/25/startling-word-discovery/#comments</comments>
		<pubDate>Fri, 25 May 2012 07:32:59 +0000</pubDate>
		<dc:creator>Martin Roberts</dc:creator>
				<category><![CDATA[fun]]></category>
		<category><![CDATA[text]]></category>

		<guid isPermaLink="false">http://blog.zarang.com/?p=4</guid>
		<description><![CDATA[My eight year-old daughter came home the other day and wrote the word &#8216;startling&#8217; on a piece of paper and put it in front of my face. I have come to learn that this is her way of saying, &#8220;Daddy, let&#8217;s play this word game that I learnt at school! Oh, and by the way, [...]]]></description>
			<content:encoded><![CDATA[<p>My eight year-old daughter came home the other day and wrote the word &#8216;startling&#8217; on a piece of paper and put it in front of my face. I have come to learn that this is her way of saying,</p>
<blockquote><p>&#8220;Daddy, let&#8217;s play this word game that I learnt at school!<br />
Oh, and by the way, let&#8217;s skip the boring part where I tell you the rules.<br />
Instead let&#8217;s just play and when you make a mistake I&#8217;ll tell you.&#8221;</p></blockquote>
<p>I&#8217;ll be a bit kinder and explains the rules at the start: You have to pick a letter so that the remaining letters still form a word. (So in the first round, I could have picked the letter &#8216;t&#8217;, or &#8216;l&#8217;, which would result in the words &#8216;starling&#8217; or &#8216;starting&#8217;, respectively.)</p>
<p>You win the game if you correctly identify a sequence of letters to eliminate at each state, so that your last valid word is only a single letter long. For example, here is one way you can produce a winning sequence:</p>
<p><strong>startling, starting, staring, string, sting, sing, sin, in, I.<br />
</strong><br />
She then said that her teacher had told the class that this was the longest word where you can go all the way down to 1-letter. Being as curious as I am, I thought I would try to verify this. Furthermore, I was intriguied as to what other words had this characterisitic.</p>
<p>Over the last two decades I have mainly used Mathematica for its unparalleled numerical and algebraic functionality (eg I used it quite extensively during my honours and PhD years in theoretical physics.) However, more recently I have been trying to leverage some of the non-mathematical functionality in it. I thought that this was an execellent excuse to use some of its English language capabilities.</p>
<p>Ever since v6.0, the word data has now been integrated into the core kernel of Mathematica, so you don&#8217;t need to import any additional packages. Although, there are some incredibly rich functions such as WordData which can output high-levels of word data. In this case though, I will just use the core function is DictionaryLookup[w] which returns w if the input string is found in the dictionary, and a null list otherwise.</p>
<pre class="brush: plain; title: ; notranslate">
vocabulary = DictionaryLookup[];

isWordValid[myWord_] := isWordValid[myWord] =
  DictionaryLookup[myWord, IgnoreCase -&gt; True] != {};

wordParts[myWord_] := wordParts[myWord]=
  Table[StringDrop[myWord, {i}], {i, StringLength[myWord]}];
</pre>
<p>The first thing to note is the slightly unusual syntax</p>
<pre class="brush: plain; title: ; notranslate">
myFunction[args_] := myFunction[args] = returnSomething
</pre>
<p>instead of the simpler and far more common</p>
<pre class="brush: plain; title: ; notranslate">
myFunction[args_] := returnSomething
</pre>
<p>The difference is that the latter syntax will evaluate <em>myFunction</em><br />
each time, whereas the first format will store the value (in <em>myFunction</em>[.]) and use this value the next time it is invoked. The most useful time to use this syntax is when repeated unchanging evaluations are required which is very common in recursive definitions like this.</p>
<p>The second thing to notice is that <em>wordParts </em>keeps all the remaining letters in their original order. There are other variants of this spelling challenge where you are allowed/required to rearrange the letters. In this case, the following function would product the requisite set of possible words.</p>
<pre class="brush: plain; title: ; notranslate">
wordPartsWithRearrangement[myWord_] :=
  wordPartsWithRearrangement[myWord] =
    Map[StringJoin,
      Permutations[
        Characters[myWord],{StringLength[myWord]-1}
       ]
    ]
</pre>
<p>The essential part to this particular solution is creating a recursive function <em>findSequences</em> that finds all full-length (ie. winning) sequences.</p>
<p>Then, just to speed things up we limit ourselves to trying to find the long words that have full-length sequences. Due to the recursive algorithm that we used, we result is a tree with each branch being of varying lengths (all the non-null ones are full length), so we then flatten it out. We could either flatten it by looking at its structure and taking all non-null elements that are of full-length, however, I thought the simplest technique was to exploit the fact that I have made every proper sequence end in the &#8220;!&#8221; character.</p>
<pre class="brush: plain; title: ; notranslate">
findSequences[wordPath_, myWord_] :=
  findSequences[wordPath, myWord] =
  Module[
   {newPath},
   newPath = Join[wordPath, {myWord}];
   If[ StringLength[myWord] == 1,
    Join[newPath, {&quot;!&quot;}],
    Map[findSequences[newPath, #] &amp;,
     Select[wordParts[myWord], isWordValid[#] &amp;]]
    ]
   ];
</pre>
<p>&nbsp;</p>
<pre class="brush: plain; title: ; notranslate">
flattenTree[myTree_] :=
  Extract[myTree,Map[Drop[#, -1]&amp;,Position[myTree,&quot;!&quot;]]] ;

fullSequence[myWord_] := fullSequence[myWord] =
  flattenTree[findSequences[{}, myWord] ]
;
</pre>
<p>&nbsp;</p>
<pre class="brush: plain; title: ; notranslate">
minimumWordLength = 9;
longWords =
  Select[vocabulary, StringLength[#] &gt;=  minimumWordLength &amp;];
universalList =
  Flatten[Map[fullSequence,longWords],1];
</pre>
<p>The <em>universalistList </em>indicates that there are 52 nine-letter words that have full sequences, however, a typical one looks like this:</p>
<p><strong>threadier, threader,treader, trader,trade,trad,tad,ad,a,!<br />
</strong></p>
<p>At first glance, it appears that there was mistake in my code, given some of these words. However, if you grab look in a dictionary (or better still, use Mathematica&#8217;s WordData[.] function!) you will actually find all of these words.</p>
<p>So although some people will be adamant that this is a fully correct soltution, I was also pragmatic about the idea that I wanted to find other words that her Primary School teacher could have used instead.<br />
I don&#8217;t think that she, or the kids would think that the word &#8216;trad&#8217; was reasonable. So what follows is a fully subjective list of words that I would think would be unfair, inappropriate, etc to expect or want the child to use in finding a solution. As I said, I fully acknowledge that this list is subjective and maybe you and your kids are smarter than me, and that&#8217;s fine by me.</p>
<pre class="brush: plain; title: ; notranslate">
uncommonWords = {
  &quot;ac&quot;, &quot;ad&quot;, &quot;ans&quot;, &quot;aping&quot;, &quot;ashe&quot;, &quot;ates&quot;, &quot;bi&quot;,   &quot;chi&quot;, &quot;chis&quot;, &quot;dis&quot;, &quot;downing&quot;, &quot;eking&quot;,   &quot;fan&quot;, &quot;fran&quot;, &quot;id&quot;, &quot;ines&quot;, &quot;ins&quot;,  &quot;la&quot;, &quot;lan&quot;, &quot;latte&quot;, &quot;lea&quot;, &quot;lin&quot;,   &quot;mi&quot;, &quot;mit&quot;, &quot;miter&quot;,   &quot;oping&quot;,  &quot;pas&quot;, &quot;pater&quot;, &quot;paters&quot;, &quot;pates&quot;, &quot;peaces&quot;,   &quot;peking&quot;, &quot;pieing&quot;, &quot;piing&quot;, &quot;piking&quot;, &quot;platte&quot;,  &quot;prattlers&quot;,&quot;prig&quot;, &quot;raping&quot;, &quot;rappings&quot;, &quot;ratters&quot;,   &quot;roweling&quot;, &quot;rowling&quot;,  &quot;san&quot;, &quot;sate&quot;, &quot;sating&quot;, &quot;sine&quot;, &quot;sines&quot;, &quot;siting&quot;,   &quot;slitter&quot;,&quot;spritzes&quot;, &quot;stalkings&quot;, &quot;stan&quot;, starvings&quot;,&quot;stringer&quot;, &quot;stringers&quot;, &quot;stringier&quot;,   &quot;ta&quot;, &quot;tam&quot;, &quot;tate&quot;, &quot;ti&quot;, &quot;whit&quot;, &quot;withe&quot;, &quot;withs&quot;
};

usesOnlyCommonWords[mySequence_] :=
  Intersection[uncommonWords, ToLowerCase[mySequence]] == {};

finalSequences = Select[
  universalList, usesOnlyCommonWords[#] &amp;
];

finalWordList  = Union[finalSequences[[All, 1]] ];
</pre>
<p><strong>Final Result<br />
</strong><br />
If you are happy to include the word &#8216;siting&#8217;, then &#8220;splittings&#8221; is actually the longest word with this special characteristic.</p>
<p>However, if you don&#8217;t &#8216;like&#8217; this word, then the teacher was right <strong>&#8220;startling&#8221; really is the longest word</strong>. However, with that said, we now know that there are four other equal winners: <strong>cleansers, roadsters, splatters, and starlings</strong>.</p>
<p>I gave these 4 additional words to my daughter and she solved them in about five minutes &#8211; much less time than I took to write the Mathematica code, and far less time that I took to write this post. But I am happy, and thought it a worthwhile and enjoyable exercise for all.</p>
<p>___</p>
<p>If you liked this post, please join me at <a title="Google+" href="https://plus.google.com/u/0/114742749367502490935/posts">Google+</a> or <a title="Quora" href="http://www.quora.com/Martin-Roberts">Quora</a>.<br />
For more information about me, visit <a title="LinkedIn" href="http://www.linkedin.com/in/martinroberts">LinkedIn </a>or <a title="here" href="http://martinroberts.brandyourself.com/">here</a>.</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.zarang.com/2012/05/25/startling-word-discovery/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>
