GPU

This is specialized electronic circuit created for manipulating images rapidly. ie Specialized in Graphics & Image processing.
GPU has 1000's of cores making them great for small computations done in parallel.
Not good for serial tasks.
In 2018 AMD/ATI(33%) & Nvidia(66%) captured 100% of GPU market.

Applications

Used game consoles, Create high resolution images, videos.
In machine training, deep learning neural networks, GPUs can be 250 times faster than CPUs.
Complex mathematical calculations, financial simulations and risk modeling
Encryption, Decryption or processing of network traffic.

CPU vs GPU

CPU GPU
Cores 4 or 8 Nvidia's GeForce:4000, AMD's Radeon: 2500, Intel's HD 500: 1100
Good For tasks Serial Parallel
Costing Same same
Transistors does data caching and flow control data processing

Graphics Card

GPU is fitted on Graphics Card, which fits on PICe Slot. Though GPU can be fitted on motherboard directly as well.
Graphics Card / Graphics Adapter is used in gaming PC or a professional high-performance PC for graphic processing.
VRM(Voltage Regulator Module)/(PPM)Processor Power Module/Voltage Regulator:
This gives power to the GPU. It converts 12V to around 1V to 1.5V (approx) which is normally the voltage level at which GPU operates. No of VRMs on a graphics card varies from card to card. VRMs can get very hot & require good cooling.

Cooler:
Every graphics card comes with Cooler to keep the temperature of GPU, VRAM, and VRM down to safer levels. Some of the high-end graphics cards also come with Liquid / Water Cooling or Hybrid Cooling Solution

PCB(Printed Circuit Board)
Base or board where all the components including GPU, VRAM, VRM, Capacitors, Sensors are mounted. High-end graphics cards have more components so they require larger PCB compared to entry-level graphics cards.

Connectors of Graphics Cards
PCI Express x16 Connector: The interface through which graphics cards communicate with the motherboard and processor.
6-Pin & 8-Pin PCI-E Connectors: Usage Needed when Graphics cards need higher power. This connector provides external power from the PSU.
Graphics Card

DGPU (Dual GPU)

Graphics card having 2 GPUs present on their PCBs. Dual graphics card as a combination of two similar graphics cards in SLI or Crossfire setup on a single PCB.
limited to some games and applications but not all games are optimized for dual GPUs.
Examples:
AMD (Radeon HD 6990, Radeon HD 7990, Radeon R9 295X2, Radeon Pro Duo, Radeon Pro Duo (Polaris)), Nvidia(GeForce GTX TITAN Z, GeForce GTX 590)