the Gwent tournament is not vital but you ought to help Triss save the mages of Novigrad before you leave. These include multiple short story side-quests, a handful of Witcher Contracts, Hidden Treasure challenges and a. Traveling to Skellige itself from Novigrad for the first time does not necessarily advance the 'time' of the overall world, and you can quickly flit back to Novigrad if you want to take care of business there, but it's worth dealing with what would be considered the main quests of Novigrad before heading on, e.g. There are numerous side-quests available in the expansive lands of Toussaint.
I shan't spoil what these moments are, but the game will be pretty obvious in prompting you about supposed 'points of no return'. I'd consider most things you get from bulletin boards to be 'essential' to the game's plot. These include multiple short story side-quests, a handful of Witcher Contracts, Hidden Treasure challenges. Question mark icons on the map don't matter too much. There are numerous side-quests available in the expansive lands of Velen (No Mans Land). Several secondary quests factor greatly into the main story (an early example in Velen is finding the magic lamp for Keira that will lead you down a very long tangent which factors into whether she joins you at Kaer Morhen in the end game, or attempts to reason with Radovid and meets a rather horrible end). Witcher contracts can impact the outcome of at least one secondary quest.