[Wranglers] Stream / changefeed that only emits columns that have changed in a record.
Ali, Saqib
docbook.xml at gmail.com
Wed May 7 17:19:10 UTC 2025
Hi,
I am looking for a way to design a stream that ONLY emits columns that
have changed in a record. If I use the stream, it emits the entire
record even if only one of the column values has changed in that
record.
Source table is as following:
create or replace table web_users (user_id int, name string, state string);
insert into web_users values (1, 'saqib', 'CA');
insert into web_users values (2, 'ali', 'AZ');
update web_users set state = 'MN' where user_id = 1; -- sample update
One approach is that I unpivot (transpose) this data into Key Values
as following and build a stream on it:
create or replace view users_pivoted as (
select *
from web_users
unpivot include nulls (kv for k in (name, state))
);
create or replace stream users_pivoted_stream on view users_pivoted;
But I get the following error:
Change tracking is not supported on queries with 'UNPIVOT_CLAUSE'.
Any other way I can emit changes at the column level?
More information about the Wranglers
mailing list