with ptb_word_lm.py, there is a bug dealing with cell state for the case when there is more than one hidden layer.
initialState is like state which is a return value of cell(...)
it has nLayers "LSTMStateTuple (c, h)"s: that is,
len(state) = nLayers, len(state[i]) = 2
c and h are tensors of shape [batchSz, hiddenSz], respectively
c and h are also aliases of 0 and 1, respectively
don't confuse c and h as aliases with c and h of (c, h)
feed_dict[c] = state[i].c # state[i].c indicates c of (c, h)
feed_dict[h] = state[i].h
print("feed_dict[c].shape=", feed_dict[c].shape)
print("feed_dict[c]=", feed_dict[c])
feed_dict[c] is a tensor of [batchSz, hiddenSz]
The solution is as follows: