Input and embedding layer architecture