To compute fees we need the median block weight for 100 blocks (AFAIK). I was wondering how exactly is the block weight computed? Is it just the sum of weights of all of its transactions? Or are other things also used to compute it (like maybe the blockheader)?
The block weight is indeed the sum of the transactions it contains (including the coinbase transaction). For most transactions, the weight is the size in bytes. For transactions with more than 2 outputs and bulletproofs, weight is adjusted up, see details in get_transaction_weight in src/cryptonote_basic/cryptonote_format_utils.cpp.
You do not normally need to compute fees yourself, this will be done automatically. Moreover, recent versions of Monero will return a new transaction's weight via RPC (as well as its fee).