In these older chips⁰ such instructions are undocumented because they were not intended to exist, rather than they were but did not work in all circumstances.
It is a side effect of how the instructions are decoded into what parts of the CPU are involved: if this bit is set then affect the X register, unless this other bit is set in which case affect Y, and if one of these bits is set use the ALU, … This results in other instructions magically appearing because it isn't a case of every valid instruction being directly listed¹ and every invalid one having no definition at all. It is also why the opcodes seem strewn around the instructions space arbitrarily, sometimes in obvious groups and sometimes apparently spread widely. And why it is risky to use them: if a later revision of the CPU adds any new instructions, the accidentally useful opcodes could become something quite² different as the designers only need to make sure the official ones keep functioning as they did before.
--------
[0] And probably more current ones too, though I've not kept a close eye on CPU design like the university aged and younger versions of me used did.
[1] There usually isn't a straight table of “this code means do that to that with this and store there” — that would take more silicon, and likely be notably slower, than the pass-the-bits-through-a-bunch-of-logic-gates approach.
There isn't any dedicated instruction decoding for these "illegal" instructions, at all. Generally, just two regular instructions happen to be executed at once. Some of these work out reliably and somewhat useful, others result in the processor becoming stuck (so-called JAM or KIL instructions). Some others kind of work, but don't yield any external result (much like a multi-byte and multi-cycle NOP), e.g., if we try to what would amount to store a register with immediate address mode. (Storing doesn't set any flags and there is no viable write address, therefore there isn't any external result to this operation.)
Generally speaking, there are no legal opcodes with both lowest bits set and the decoding triggers for both instructions with either the lowest or the second lowest bit set. (There are some more outside of this pattern, but these result more often in a "jammed" CPU than not.)
Maybe that was the reason they were undocumented?