Complex matrix equation involving complex numbers how to solve

SOURCE: how to solve equation involving angles

use complex mode

SOURCE: In complex number mode, how to get out?

"Is there anything that I can do about this to stop my answers being complex numbers when solving cubic equations?"
You are making an assumption that is unwarranted.Namely that a cubic equation must necessarily have 3 real roots. This is not the case.

Let your equation be

If coefficients are complex you should expect some complex roots. Right?
If the coefficients are REAL then depending on the discriminant

you can have three cases
DELTA positive : three distinct real roots
DELTA=0 , the equation has a multiple root and all roots are REAL
DELTA negative: the equation has ONE real root and 2 complex roots that are complex conjugate of each other.

It suffices to look at the Tartaglia/Cardano expressions of the roots to know that more often than not there is going to be complex roots.

SOURCE: Hello How to solve complex

The simultaneous equation solver requires the coefficients to be real. Similarly matrices must have real coefficients.
Your only alternative is to express each of A and B as a real part and imaginary part.
A= x1+iy1
B=x2+iy2.
Substitute x1+iy1 for A in the two equations. Substitute x2+iy2 for B in the two equations. Do the algebra. Gather real parts and gather imaginary parts. Split each original equation into two equations: One equation comes from setting Real Part of left side = real part of right side (1); the other equations comes from setting the imaginary part of left side= imaginary part of right side (here 0).

Do the same procedure for the 2nd original equation.
At the end of the process you will have 4 coupled linear equations in the 4 unknowns (x1,y1,x2,y2).

Then you might want to use the calculator to solve the derived system. Once you have x1,y1,x2,y2 you reconstruct A=x1+iy1, etc.

SOURCE: solving linear equations in 3

Press key Mode, first select option 5:EQN (by pressing key 5), then option 2: a_nX+b_nY+c_nZ=d_n (by pressin 2).
Now input coefficients from equations in table (each column is for one variable), just type in number and press =, then use arrows to move to another cell.

SOURCE: Edit Casio FX - 9860G SD have complex equations??

Thhe Casio FX-9860G SD can solve a polynomial equation of degree 2 or 3 with REAL coefficients. If the complex MODE is set to REAL it will find the real roots. If the complex mode is set to a+ib, it will find the real and complex roots.

Apparently it will take coefficients that are real, and will give a Ma Error if any coefficient is complex.

Addendum.
The calculator CANNOT solve equations with complex coefficient. YOU can however convert the system of linear equations with ccomplex coefficients ( of the type you show) as a system of 4 linear equations in 4 unknowns; Split x into a real and an imaginary part, split y into a real and an imaginary part. Substitute Real(x)+iIm(x) for variable x in the equations; substitute Real(y)+iIm(y) for y in the two equations; do the algebra. In each of the original equations split the Real and imaginary parts. You should be able to derive 4 linear equations in unknowns Real(x), Im(x), Real(y), and Im(y).
Use the linear equation solver to obtain the solutions. Recompose x=Real(x)+iIm(x), and y=Real(y)+iIm(y)

Alternatively, after you create the system of 4 linear equations you can use the matrix utility to find Real(x), Im(x), Real(y) and Im(y) and recompose the x and y.