When a layer of liquid condenses on the inside of a cylindrical tube, it is subject to capillary instability, first investigated in the classical works of Rayleigh and Plateau. The same instability is responsible for the break-up of liquid-liquid interfaces in the well-known core-annular flow configuration. Modeling capillary instability is essential for two emerging applications of microscale systems. In photonics, capillary condensation in elongated pores is used for tuning optical properties of mesoporous silicon and interfacial instabilities have to be avoided. In biomicrofluidics, the break-up of a confined viscous jet is a useful tool for encapsulation of individual living cells by liquid droplets. In both applications, the unstable fluid interface is confined to a channel of polygonal cross-section, not a cylindrical tube. A mathematical model of capillary instability for such configuration is the subject of the talk. Linear stability criteria are established and the effect of geometry of the confining walls on the strongly nonlinear evolution of the fluid interface is investigated.