To count words in a newline-separated dictionary, use std.mem.tokenizeScalar to create an iterator over the text. Call next() until it returns no more tokens, counting each word along the way. section 4.1.3
Listing 4.1 Counting the dictionary words
fn countWordlist(text: []const u8) usize {
var count: usize = 0;
var word_iter = std.mem.tokenizeScalar(u8, text, '\n'); #1
while (word_iter.next()) |_| { #2
count += 1;
}
return count;
}
This function splits a byte slice on '\n' and counts the resulting words. section 4.1.3
The function is generic: its comptime type parameter T determines the buffer element and delimiter types. It returns a TokenIterator(T, .scalar). The function body must be valid for the supplied type. section 4.1.3
Listing 4.2 Comptime type parameters for fun and profit
pub fn tokenizeScalar(
comptime T: type, #1
buffer: []const T, #2
delimiter: T, #3
) TokenIterator(T, .scalar) #4
The declaration ties the buffer and scalar delimiter to the same element type. section 4.1.3
The same iterator can parse a compile-time wordlist into an array. The parser checks that each word has four characters and reports a compile error otherwise. section 4.1.3
Listing 4.4 Parsing the wordlist into an array
fn parseWordlist(
comptime wordlist: []const u8, #1
) [countWordlist(wordlist)][4]u8 { #2
var list: [countWordlist(wordlist)][4]u8 = undefined;
var i: usize = 0;
var word_iter = std.mem.tokenizeScalar(u8, wordlist, '\n');
while (word_iter.next()) |word| : (i += 1) { #3
if (word.len != 4) {
@compileError(std.fmt.comptimePrint( #4
"the word '{s}' is not four characters long.",
.{word},
));
}
list[i] = word[0..4].*; #5
}
return list;
}
The parser counts entries for its array size, then tokenizes again to populate it. section 4.1.3
