concept symbol in category machine learning

This is an excerpt from Manning's book Machine Learning with TensorFlow, Second Edition MEAP V08. Login to get full access to this book.

Perhaps the toddler version of yourself asked your dad, upon being handed a new action figure of a caped superhero, is this figure the Superman? It wasn’t, but it was close: it was another hero that looked similar. Why did you think it was the Superman? Likely the cape; the dark hair, a possible triangle near the chest with some sort of symbol in it – these image features looked familiar. Your biological neural network fired with the input image, and it retrieved the label that was reinforced over time when your parents verbally provided it to you. And now, having been handed a new toy, you spit out the label, and your parents correct you: “No honey, it’s Shazam. It looks like Superman though; I could totally understand why you thought that!” Boom, label added based on the slightly different features and you move on to learning more!

Figure 18.6 A mapping from symbols to scalars

The following snippet shows how to define such a mapping between symbols and numeric values using TensorFlow code:

It’s also typical to convert letters or words in a vector representation, often called embedding. TensorFlow provides a handy function called embed_sequence that can help you embed the integer representation of symbols. Figure 18.10 shows how the encoder input accepts numeric values from a lookup table. You can see it in action at the beginning of listing 18.13.

Figure 18.10 The RNNs accept only sequences of numeric values as input or output, so you’ll convert your symbols to vectors. In this case, the symbols are words, such as the, fight, wind, and like. Their corresponding vectors are associated in the embedding matrix.

Listing 18.19 Training the model

input_sentences = load_sentences('data/words_input.txt')    #A
output_sentences = load_sentences('data/words_output.txt')  #B
 
input_seq = [
    [input_symbol_to_int.get(symbol, input_symbol_to_int['<UNK>'])
        for symbol in line]                                       #C
    for line in input_sentences                                 #D
]
 
output_seq = [
    [output_symbol_to_int.get(symbol, output_symbol_to_int['<UNK>'])
        for symbol in line] + [output_symbol_to_int['<EOS>']]         #E
    for line in output_sentences                                    #F
]
 
sess = tf.InteractiveSession()
sess.run(tf.global_variables_initializer())
saver = tf.train.Saver()                                          #G
 
for epoch in range(NUM_EPOCS + 1):                                #H
 
    for batch_idx in range(len(input_sentences) // BATCH_SIZE):     #I
 
        input_data, output_data = get_batches(input_sentences,      #J
                                              output_sentences,
                                              batch_idx) 
 
 
        input_batch, input_lenghts = input_data[batch_idx]
        output_batch, output_lengths = output_data[batch_idx]
 
        _, cost_val = sess.run(                                  #K
            [train_op, cost],
            feed_dict={
                encoder_input_seq: input_batch,
                encoder_seq_len: input_lengths,
                decoder_output_seq: output_batch,
                decoder_seq_len: output_lengths
            }
        )
 
saver.save(sess, 'model.ckpt')
sess.close()

copy

This is an excerpt from Manning's book Machine Learning with TensorFlow. Login to get full access to this book.

It’s also typical to convert letters or words in a vector representation, often called embedding. TensorFlow provides a handy function called embed_sequence that can help you embed the integer representation of symbols. Figure 11.10 shows how the encoder input accepts numeric values from a lookup table. You can see it in action at the beginning of listing 11.13.

Figure 11.10. The RNNs accept only sequences of numeric values as input or output, so you’ll convert your symbols to vectors. In this case, the symbols are words, such as the, fight, wind, and like. Their corresponding vectors are associated in the embedding matrix.

Listing 11.20. Training the model

input_sentences = load_sentences('data/words_input.txt') 1 output_sentences = load_sentences('data/words_output.txt') 2 input_seq = [ [input_symbol_to_int.get(symbol, input_symbol_to_int['<UNK>']) for symbol in line] 3 for line in input_sentences 4 ] output_seq = [ [output_symbol_to_int.get(symbol, output_symbol_to_int['<UNK>']) for symbol in line] + [output_symbol_to_int['<EOS>']] 5 for line in output_sentences 6 ] sess = tf.InteractiveSession() sess.run(tf.global_variables_initializer()) saver = tf.train.Saver() 7 for epoch in range(NUM_EPOCS + 1): 7 for batch_idx in range(len(input_sentences) // BATCH_SIZE): 9 input_data, output_data = get_batches(input_sentences, 10 output_sentences, batch_idx) input_batch, input_lenghts = input_data[batch_idx] output_batch, output_lengths = output_data[batch_idx] _, cost_val = sess.run( 11 [train_op, cost], feed_dict={ encoder_input_seq: input_batch, encoder_seq_len: input_lengths, decoder_output_seq: output_batch, decoder_seq_len: output_lengths } ) saver.save(sess, 'model.ckpt') sess.close()

copy

11.3. Vector representation of symbols

Words and letters are symbols, and converting symbols to numeric values is easy in TensorFlow. For example, let’s say you have four words in your vocabulary: word₀: the; word₁: fight; word₂: wind; and word₃: like.

Figure 11.6. A mapping from symbols to scalars

The following listing shows how to define such a mapping between symbols and numeric values using TensorFlow code.