Ryan first you want to go back and double (or triple0 check your work. Always make sure that everything is properly connected and back in place the way it should be. If you removed the CPU to apply new thermal past, check to make sure it is properly inserted etc. If you did not remove it, it'll be okay. Next, you want to check the power supply output. I know you already know this, but LED 1 coming on and LED2 not coming on is still non-specific. It can be the power board or the logic board.
If the second LED is not visible when the power button is pressed:
Verify that the power button is connected to the power supply.
Verify power button functionality.
Verify the cable connection between the power supply and the logic board.
Verify the power supply.
Verify the logic board.
Of course you already tried to reset the SMC and PRAM but, just in case:
Resetting the System Management Controller (SMC)
The System Management Controller (SMC) is a chip on the logic board that controls all power functions. If the computer is experiencing any power issue, such as not starting up, not displaying video, sleep issues, or fan noise issues, resetting SMC may resolve it.
Intel-based Macs: Resetting the System Management Controller (SMC).
Note for iMacs: If you press the power button while inserting the power cord, the iMac will enter a mode in which the fans run at full speed.
Resetting Parameter RAM (PRAM)
PRAM stores certain system and device settings in a location that OS X can access quickly. Exactly which settings are stored in the computer’s PRAM varies depending on the type of computer as well as the types of devices and drives connected. To reset PRAM:
1. Shut down the computer.
2. Locate the following keys on the keyboard: Command, Option (Alt), P, and R. You will need to hold these keys down simultaneously in Step 4.
3. Press power button.
4. Immediately press and hold Command-Option-P-R keys.
Important: You must press this key combination before the gray screen appears.
5. Hold down keys until computer restarts, and you hear startup chime a second time.
6. Release keys.
Right now it sounds to me like a bad power supply. See if you have 12V coming of it. That would correspond with LED #2
Measure this on the logic board connector where the power supply connects to:
Pin 1==> GND
Pin 2==> GND
Pin 3==> GND
Pin 4==> PP12V ACDC
Pin 5==> PP12V ACDC
Pin 6==> PP12V ACDC
I just noticed that when the iMac power button is pressed, the fan spins to life for around two seconds and shuts off. Then, when you unplug it the fan spins again and stops. Don’t know if this will help me figure it out though.
@ryan39167 - That’s a good sign of a POST issue. As you started off with a working system, I really would check the CPU is seated properly as well as check the socket pins. As if the CPU doesn’t respond SMC will crowbar the power supply.