That light is showing that there is power to amp in that speaker but there is something wrong with the amp or the speaker inside and nothing is coming out. You will have to troubleshoot the amp and check the resistance of the speaker to see what the problem is.
The fuse is inside and if it blows, there is something wrong that caused the failure and MUST be repaired as just changing the fuse MAY do additional damage.