QuickData.Words.FSharp 0.3.0

QuickData.Words.FSharp

Contents

• Overview
• Word Generation
  • Word Length Range with code examples
  • Words with code examples
• Sentence Generation
  • Sentence Length Range with code examples
  • Sentences with code examples
• Word Dictionary
• Dependencies
• Usage

You can visit the QuickData.FSharp GitHub repository to report issues, ask questions, or make suggestions. You can also read about the changes across different versions in the release notes there.

Overview

This QuickData.Words.FSharp package contains the QuickData.Words.FSharp namespace for F# developers which provides some types, modules, and their related functions for building and manipulating sequences of English words and 'nonsense' English sentences.

All of the sequences generated by the functions in this package are standard F# sequences, and all of the internal processing is done exclusively with standard F# sequences, so they give you all of the benefits of, and follow the same rules as, standard F# sequences.

The modules provided are:

• WordLengthRange : Defines the possible number of characters (lengths) for the words which are to be generated;
• Words : Defines functions which can be used to create sequences of words (strings) which are chosen from the internal dictionary;
• SentenceLengthRange : Defines the possible number of words for the sentences which are to be generated;
• Sentences : Defines functions which can be used to create sentences which are strings containing words, each separated by spaces, with a full stop (period) at the end.

Word Generation

Word Length Range

The WordLengthRange type defines a range to be used to create words.

A WordLengthRange is specified with one or more int values, each of which determines a possible word length.

A WordLengthRange can be one of three cases:

• LengthBetween : Where two different lengths - minimum and maximum - were used to create it (the usual way to do it);
• LengthChoices : Where a collection of more than one (possibly non-contiguous) length was specified to create it;
• OneLengthOnly : Where the same length was used twice, or only one length was specified, to create it.

The WordLengthRange module provides functions for working with a WordLengthRange, including:

• fromInts : Creates a WordLengthRange from the two (preferably different) provided lengths;
• fromSingleInt : Creates a WordLengthRange from a single provided length;
• fromCollection : Creates a WordLengthRange from a (possibly non-contiguous) collection of lengths;
• lowValue : Returns the lowest length of the range (lowest possible number of letters);
• highValue : Returns the highest length of the range (highest possible number of letters);
• allValues : Returns a sequence containing all the possible lengths in the range.

The minimum word length is two letters and the maximum word length is twenty letters. The length(s) provided when creating a range are clamped between these values.

When creating a range from a collection of lengths any duplicate lengths are ignored so, for example, specifying lengths of 3 and 4 and 4 again will result in a range of only 3 and 4.

Some pre-prepared ranges, such as onlyLongWords or onlyEvenLengthWords, are also available for convenience.

Notes:

1. See the Word Dictionary section below for more information about the available words.

2. Depending on how many words you generate, you might not get a word of every length in the range. For example, if you specify a range of four to fifteen words (a range of twelve possible lengths) and only generate six words then you will only get six words and words of some possible lengths will not be generated.

Word Length Range Code Examples

let betweenFiveAndEightLetters = 
    WordLengthRange.fromInts 5 8 
    // -> LengthBetween (5, 8)

let longestPossibleWord = 
    betweenFiveAndEightLetters 
    |> WordLengthRange.highValue // -> 8

let sixLettersOnly = 
    WordLengthRange.fromSingleInt 6 
    // -> OneLengthOnly 6

let fromCollection = 
    seq { 3; 4; 6; 8 } 
    |> WordLengthRange.fromCollection 
    // -> LengthChoices (set [3; 4; 6; 8])

Words

The Words module provides functions for generating English words, including:

• random : Builds a sequence of words, each of which has a length within the specified range;
• cycled : Builds a sequence of words, each of which has a length within the specified cycled range;
• tombola : Builds a sequence of words, each of which has a length within the specified shuffled range;
• singleRandomWord : Returns a single randomly-chosen word, of any length, from the dictionary;
• toSentence : Returns a sentence (string) containing the source words in the order in which they are given;
• toSortedSentence : Returns a sentence (string) containing the source words in ascending aphabetical order.

A sentence, as generated here, is a string containing strings, each separated by a space, with a full stop (period) at the end. If the collection of words is empty then an empty string will be created.

Words Code Examples

let rng = System.Random 3921 // Randomly-chosen seed.

let singleWord = Words.singleRandomWord rng // -> e.g. "acquaintance"

let onlyThreeLetters = 
    Words.random rng (WordLengthRange.fromSingleInt 3) 8 
    // -> e.g. seq { "sir"; "off"; "kid"; "cup"; "gal"; "cur"; "web"; "dip" }

let range = WordLengthRange.fromInts 3 8 

let random = 
    Words.random rng range 7 
    // -> e.g. seq { "fuel"; "ready"; "lighted"; "and"; "whined"; "idea"; "poppies" }

let cycled = 
    range
    |> Words.cycled rng 
    |> Seq.take 8 
    // -> e.g. seq { "tie"; "snug"; "nests"; "crafty"; "manager"; "together"; "raw"; "gate" }

let sentence = cycled |> Words.toSentence
    // -> e.g. "Tie snug nests crafty manager together raw gate."

let onlyOddWords = 
    Words.random rng WordLengthRange.onlyOddLengthWords 8 
    // -> seq { "tower"; "loosely"; "undisturbedly"; "tin"; "straightforwardness";
    //          "uncoiling"; "unconstitutionality"; "sympathetically" }

Sentence Generation

Sentence Length Range

The SentenceLengthRange type defines a range to be used to create sentences.

A SentenceLengthRange is specified with one or more int values, each of which determines a possible sentence length (number of words).

A SentenceLengthRange can be one of two cases:

• LengthBetween : Where two different lengths - minimum and maximum - were used to create it (the usual way to do it);
• OneLengthOnly : Where the same length was used twice, or only one length was specified, to create it.

The SentenceLengthRange module provides functions for working with a SentenceLengthRange, including:

• fromInts : Creates a SentenceLengthRange from the two (preferably different) provided lengths;
• fromSingleInt : Creates a SentenceLengthRange from a single provided length;
• lowValue : Returns the lowest length of the range (lowest possible number of words);
• highValue : Returns the highest length of the range (highest possible number of words).

The minimum sentence length is three words and the maximum sentence length is twenty words. The length(s) provided when creating a range are clamped between these values.

Some pre-prepared ranges, such as onlyShortSentences or anySentenceLength, are also available for convenience.

Sentence Length Range Code Examples

let betweenFiveAndEightWords = 
    SentenceLengthRange.fromInts 5 8 
    // -> LengthBetween (5, 8)

let longestPossibleSentence = 
    betweenFiveAndEightWords 
    |> SentenceLengthRange.highValue // -> 8

let sixWordsOnly = 
    SentenceLengthRange.fromSingleInt 6 
    // -> OneLengthOnly 6

Sentences

A sentence, as generated here, is a string containing three or more English words, each separated by a space, with a full stop (period) at the end.

The Sentences module provides functions for generating sentences, including:

• cycled : Builds a new sequence whose elements are sentences, each with a number of words within the specified sentence length range (word lengths and sentence lengths are cycled);
• tombola : Builds a new sequence whose elements are sentences, each with a number of words within the specified sentence length range (word lengths and sentence lengths are shuffled);
• singleRandomSentence : Returns a single sentence containing a random number of randomly-chosen words from the dictionary.

Sentences Code Examples

let rng = System.Random 2345 // Randomly-chosen seed.

let wordLengthRange = WordLengthRange.fromInts 3 8 

let sentenceLengthRange = SentenceLengthRange.fromInts 4 7 

let tombolad = 
    Sentences.tombola rng wordLengthRange sentenceLengthRange

let tomboladFirstFive = 
    tombolad
    |> Seq.take 5 
    // -> e.g. seq { "Eyes piteous cub choir summoned suites."; 
    //               "Map pounced included smoky crab untidy ashy."; 
    //               "Trot oat hunch executes."; "Sorts return weed panorama starved."; 
    //               "Puffins seemed drop consider carol cub." }

let tomboladNextTwo = 
    tombolad
    |> Seq.take 2 
    // -> e.g. seq { "Meeting manger probably throw."; 
    //               "Costly cure cling alarmed scribble day hid." }

let singleRandomSentence = 
    Sentences.singleRandomSentence rng wordLengthRange
    // -> e.g. "Eel flashed pop woollen gas act."

Word Dictionary

To compile the dictionary in this package I first extracted (in a fairly crude manner) the words from the raw text (including headers and footers) of the following books from Project Gutenberg:

• Aesop for Children;
• Alice's Adventures in Wonderland;
• The Jungle Book;
• The Railway Children;
• The Secret Garden;
• The Wind in the Willows.

I then did a quick scan/clean-up to remove anything that was obviously, to me, not a real word (e.g "untigerish") or was not appropriate for use with this package.

I then removed all words with fewer than two letters, or more than twenty letters, and those which contained any non-letter characters, e.g. "o'clock", "self-contradictory", etc.

I then added some colours, the NATO phonetic alphabet, and some extra-long words.

There is a mix of British English (e.g. flavour) and American English (e.g. flavor) words in the dictionary.

Some of the words are peoples' names and some of the words are place names (and some company/organisation names might also have made it through my checks when I wasn't being too careful).

I cannot guarantee that none of the words are offensive or controversial in some way. (I can't easily check all 11,000+ words manually, and I can't know what's potentially offensive/controversial to everyone, but I've removed any obviously 'distasteful' words which I noticed.)

While there are no duplicate words in the dictionary, it is statistically possible that you might get the same word randomly generated more than once in any result. This will be especially true if you specify a large high value for WordLengthRange (or use WordLengthRange.anyWordLength) as there are far fewer very long words so they will come up a lot more often. It's best to keep the high value of the range to less than thirteen for better results.

Note: Because the dictionary might change through additions, removals, and/or modifications, you should not rely on the generated words being the same, even with the same input and random number generator seed, between different versions of the package. Changes to the dictionary will not be mentioned in the release notes.

Library (DLL) Size

The library (DLL) installed by this package is quite large (just under 400KB) because the dictionary is stored in a very basic way with no compression. I will be considering compressing the dictionary to make the DLL smaller in the future but this cannot be relied upon. You will need to make a decision as to whether the size of the DLL is acceptable for your purposes.

Dependencies

This package is dependent upon FSharp.Core which you will be using anyway, and the QuickData.Core.FSharp package which will normally be automatically installed if you install this package.

Usage

The types and functions in this package have been designed to be used only with F#.

However, they may also be usable with C# but this has not been tested, so use them with C# at your own risk.