1. EU Content Geographic Filter Not Enabled So the way content gets into labeling queues is based on language. If someone is streaming in Arabic, it goes to the Arabic queue. Doesn't matter where they are physically. That means if someone is in Germany but streaming in Arabic, their content goes to the Arabic queue, which is worked by Majorel in Morocco. So EU user data is being processed outside the EU. The platform actually has a control for this. It can detect when content originates from an EU user and automatically pull it out of a non-EU queue before any labeler sees it. It just removes it. This control is live and working on other queues in the platform. Issue Labeling, Commercial Content, Clickbait — all have it turned on. Tens of thousands of EU tasks get caught and removed every month on those queues. On Live AIM and VQA queues — zero. The control has never fired. Not once across the entire audit period. We looked into why. Every queue has a classification field called "entity." The queues where the control works are all entity 1. Every Live AIM queue is entity 9. The control is only scoped to entity 1. Entity 9 was never included. So the control exists, it works, it's deployed elsewhere in the same platform — it just doesn't reach ADSO queues because of how they're classified. In terms of scale — on any given day, roughly 1,700 to 2,400 EU-origin tasks are being completed by non-EU labelers. We confirmed this down to individual Majorel workers in Morocco labeling content from users in Germany, France, Italy, Netherlands, Belgium, Sweden. There is also no equivalent filter for US-origin content. We checked — no US filter exists on any TCS queue. You need to explain context, which is worked on by Madgerau. Who are they? He doesn't know anything. The point is they go to BPO sites. What are BPO sites? You have to kind of walk him through. He doesn't have any of the context that we have. don't overdo it. He has, you know, some understanding, but it's just certain issues you need to concisely give the context as you explain it. But overall, the whole point should get to the point. This is an executive. He doesn't have time for, you know, a giant sort of, you know, stuff like, we need to get to the point and kind of clearly explain all of the details in on the way. in our concise, but easy to follow way.Mar 202 / 21. EU Content Geographic Filter Not Enabled Content gets routed into labeling queues based on language. Arabic content goes to the Arabic queue, German to the German queue, and so on. These queues are then worked by third-party labeling vendors — BPO sites — located in various countries. For example, the Arabic queue is worked by Majorel in Morocco, the English queue by Majorel in Georgia. The issue is that routing is purely language-based with no consideration for where the user actually is. So if someone is physically in Germany but streaming in Arabic, their content goes to the Arabic queue and gets labeled by workers in Morocco. EU user data leaves the EU. The platform has a control designed to catch exactly this. It detects EU-origin content sitting in a non-EU queue and automatically removes it before any labeler sees it. This control is active and working on other queues in the platform — tens of thousands of EU tasks get caught and removed every month. On ADSO labeling queues — both Live AIM and VQA — this control has never fired. Zero activations across the entire audit period. The reason is a configuration gap. Each queue has a classification field called "entity." The EU filter only applies to queues classified as entity 1. All ADSO queues are classified as entity 9. The filter was never extended to cover them. In terms of scale, roughly 1,700 to 2,400 EU-origin tasks per day are being labeled at non-EU BPO sites. We confirmed this down to individual labelers in Morocco processing content from users in Germany, France, Italy, Netherlands, Belgium, and Sweden. There is no equivalent filter for US-origin content either. No US filter exists on any queue in the platform. go to the next one. So this issue is, you know, about the, you know, say that there's a similar control, just the way there's a control above. There's a similar control that should be working for private content...concisely explain the issue, because you've already given some context, no need to repeat yourself, but concisely explain the issue.Mar 202. Non-Public Content Filter Not Enabled Same pattern as the EU filter. The platform has a control that automatically removes content from private user accounts before it reaches labelers. It flags it as non-public content and abandons the task for compliance purposes. This control is active on other queues — roughly 25,000 activations per day. On Live AIM and VQA queues — again, zero activations. The control is not enabled. As a result, content from private accounts is being labeled by BPO workers. Across all 31 Live AIM Audio queues, approximately 4.7% of completed tasks involved content from private accounts — consistent across every queue, ranging from 2% to 6%. Extrapolated to the full year, that's approximately 921,000 private account tasks labeled in 2025. Next, go to the deduplication. This one is technical, so we need to explain it in a technical way. Exactly what's happening. You have the same bucket being pulled from the same way multiple times a day. If it happens once a day, no problem. But when it's happening multiple times a day, you get this issue and like clearly explain the batches, how the injection works, and then how there are no steps to pick up on this and it's happening across all the cues. And again, he needs to understand this. This is sort of like a, you know, common sort of issue you might see in an injection pipeline. So in a natural straightforward way, explain this issue.Mar 203. Duplicate Content Injection To understand this one, I need to quickly explain how content gets into the labeling queues. The R&D team runs extraction jobs that pull livestream clips from a large source table. Each job runs a query that says: give me all clips in this language, above this model confidence score, from the last 24 hours. The results get packaged up and pushed into the labeling queue. When this runs once a day, no issue. The job pulls the clips, they go into the queue, labelers work them. The problem happens when multiple extraction jobs run on the same day targeting the same queue. Each job independently queries the same source table with the same filters. The first job pulls a set of clips. The second job runs later and queries the same table — the clips from the first job are still sitting there. Nothing marks them as already taken. So the second job pulls them again. Both copies get injected into the same queue, assigned to different labelers, labeled separately, and both results go into the model training data. The pipeline has no deduplication check at any point — not at extraction, not at injection, not at the queue level. This is a common issue in data pipelines. The standard fix is a dedup function — you assign a unique identifier so anything selected by the first batch gets excluded from the second. That doesn't exist here. Smaller language markets get hit hardest because the pool of available content is smaller. When two jobs both query for Kurdish clips above a score threshold, the pool is small enough that a large proportion overlaps. Kurdish has a 32% duplication rate — nearly a third of labeling effort in that market is redundant. Hausa is at 23%. Dutch at 14%. Larger markets like Indonesian still have it, just at a lower rate — 0.5% — because the content pool is deeper and the overlap is proportionally smaller. Every single one of the 31 queues is affected. In December alone, approximately 95,000 tasks were duplicates. No queue is unaffected. The downstream impact is three things. Labeler time is wasted on content already reviewed. The model training data contains duplicate entries which can skew what the model learns. And task completion metrics are inflated because duplicate completions count as separate work. Next, I want you to go over the next one, again, focus on what we're trying to say, kind of explain that a lot of this is overlapping with the GCA team, and so we're kind of supporting them with our analysis, but this is what we saw, and kind of you can join them together, both the next two, because they have the same sort of, you know, QA pipeline-related issues is two aspects of it, so kind of clearly explain these next two observations...You're losing the concise, natural talking style room. Keep making sure you have it. You're explaining someone at a water cooler house, something works, you know, and where the issues are. Like go for a natural concise sort of explanation.Mar 202 / 24 & 5. QA Pipeline Issues These next two are related — they're both about the QA process not working as designed. There's overlap here with Mateja's work on the GCA side, so we're supporting her analysis with the technical pipeline view. Quick context on how QA works for Live AIM. There are three layers. Layer 1 is the labeling itself — a BPO worker reviews the content and submits their answer. Layer 2 is blind review — a percentage of completed tasks get automatically copied to a separate QA queue where a different person labels the same content without seeing the first answer. If those two answers disagree, it becomes a diff case and goes to Layer 3 — an in-house arbitrator who makes the final call on who was right. So what we found is that Layer 3 has stopped working entirely in six major markets — Arabic, English, Turkish, German, French, and Spanish EU. These queues used to have arbitrators — we can see completed arbitration tasks in July, August, into September. Then it drops to zero and never comes back. The arbitrators left and were never replaced. But the system doesn't know nobody is there, so diff cases keep flowing in, sitting for 30 days, and expiring. The four markets where it still works — Indonesian, Malaysian, Thai, Vietnamese — each have one or two dedicated arbitrators actively processing tasks. The six broken markets have nobody. On top of that, Layer 2 has also stopped working in four additional markets — Azerbaijani, Hebrew, Swedish, Ukrainian. These queues receive sampled tasks but there is literally no QA reviewer assigned. The system copies the first-round answer in as a placeholder, and then nothing happens. The task sits for 30 days and expires. So for these four markets, there is zero quality review at any layer. Then beyond the markets where QA is completely broken, there are eight markets that never had a QA pipeline at all. No QA queue exists for them. Chinese, Japanese, Korean, Kurdish, Nepali, Filipino, Russian, Urdu. These are not small markets — Nepali alone had 70,000 tasks in December. Combined it's about 350,000 tasks per month going straight to model training with no quality check whatsoever. And then even for the markets that do have functioning QA, a big chunk of sampled tasks are expiring before anyone gets to them. About 54,000 QA tasks expired in December across all BPOQA queues. PT EU lost 67% of its QA tasks to expiry. IT lost 41%. The review capacity just isn't matched to the volume being sampled. So when you put it all together — between markets with no QA infrastructure, markets where QA exists but nobody is assigned, and markets where QA is active but can't keep up — a substantial portion of labeling output is entering the model training pipeline without the quality oversight the project design requires. Finally, we need to explain the system access issue. So start with the roles. So you have these different types of laborers. This is what they do. And then so from an access perspective, this is what we were seeing. We looked at sort of the conning permissions and then took like a role-based access control perspective. And then this is what we were finding. And then this is what the problem is with the way they have it set up. Again, aim for clarity, maximum clarity, and try to be concise as you explain the issues. So aim for brevity while just there's a natural explanation. Well, he will clearly understand the root issue here.Mar 206 & 7. System Access Controls So there are different roles in this process. You have labelers — BPO workers who review content and submit their decisions. You have QA reviewers who independently check the labeler's work. And you have project POCs — in-house staff who own and manage the queues. Each role should only have access to what they need. Labelers should label. QA reviewers should review. And only project POCs should have visibility into broader queue data and configuration. We looked at this from two angles. First, how the queues are configured. Second, what permissions users actually have. On the queue configuration side, two issues. First is BlindQA. When a QA reviewer picks up a task, they're supposed to evaluate it independently without seeing what the labeler decided. There's a setting for this — if it's turned on, the first-round answer is hidden. If it's off, the QA reviewer can see the labeler's answer before making their own decision, which undermines the whole point of independent review. For VQA, 86% of queues don't have this turned on. For Live AIM, 13%. Second is mutual exclusion — a setting that prevents the same person from labeling a task and then QA-reviewing that same task. If this isn't on, there's nothing stopping someone from reviewing their own work. For VQA, 99% of queues don't have this enabled. For Live AIM, 100%. On the permissions side, we looked at what's actually been granted to users through the permission system. We found BPO workers with access they shouldn't have. Over a thousand BPO workers across VQA and Live AIM have a permission called "search incompleted tasks" — this lets them browse all open tasks in the queue, including ones assigned to other people and ones awaiting QA. There's no operational reason for a labeler to see other people's work. On top of that, 18 BPO workers have "view task data" permission across 12 Live AIM queues. This gives them access to completed task data and historical labeling decisions — well beyond what a labeler needs. Same issue on the in-house side. "View task data" should be limited to project POCs. We found 17 in-house users on VQA and 3 on Live AIM who have this permission but aren't project POCs. The bottom line is that the access controls aren't aligned to roles. The system doesn't enforce role-based restrictions at the queue level, and the permissions that have been granted go beyond what each role requires. need to also go over the scope so I need to go over like this is what GCA was covering and this is what GTA was covering so I need to give an overview of the scope again go for concise straightforward easy to understand very easy to understand talking points on this I need to just kind of give him an understanding of what our scope was.Mar 202 / 2Scope Overview So this audit covers two data labeling projects within ADSO — that's the team responsible for preparing training data for TikTok's content moderation AI models. The two projects are Live AIM, which deals with livestream content, and VQA, which deals with short-form video content. The audit was split between two teams. GCA covered the operational and process side — things like how projects get set up, how SOPs are drafted and shared with labelers, how labeling is actually executed, and how the QA and appeals process works day to day. GTA — our side — covered the technical and platform side. How the labeling platform is configured, how content gets injected into queues, how content gets routed, whether the right controls are turned on, and whether system access and permissions are set up properly. There's natural overlap between the two. For example, GCA looked at QA from a process perspective — are reviews being done on time, are there conflicts of interest. We looked at QA from a pipeline perspective — is the infrastructure there, are tasks actually flowing through each layer, is anyone assigned to work them. The findings reinforce each other. On the GTA side specifically, our testing covered three areas. First, the labeling platform architecture and configuration — that's the content routing, the filters, the injection pipeline, and the QA pipeline. Second, system access — who has permissions to do what on the platform. And third, change traceability — whether configuration changes to queues are logged and traceable. how do we reach out for each of our observationsMar 20How we reached each observation: EU filter / Non-public content filter — We started by looking at how content gets routed into queues and whether any controls exist to filter content based on geography or privacy status. We checked the abandon reasons across all TCS queues and found two controls actively firing on other queues — one for EU content, one for non-public content. We then checked whether these same controls fire on ADSO queues. Zero activations. We traced it back to the queue configuration and identified the entity classification as the reason the controls don't apply. Duplication — This came out of testing the injection pipeline. We were checking whether the same content was being submitted more than once by comparing the livestream room ID, clip timestamp, and policy across completed tasks within each queue. When we found matches, we checked the batch identifiers and confirmed they came from different extraction runs on the same day. We then scaled the analysis across all 31 queues and multiple days to confirm it was systematic. QA pipeline issues — We built a full pipeline trace following tasks from injection through labeling, into QA sampling, through blind review, and into arbitration. At each stage we checked whether tasks were actually being picked up and completed. That's where we found layers dropping off — arbitrators gone in six markets, no reviewers in four more, no QA queue at all in eight others. The expiry data came from checking how many sampled QA tasks were completing versus timing out. System access — We took a role-based access control approach. We pulled the permission assignments from the platform's access management table and compared what was granted against what each role should have. That surfaced the BPO workers with permissions beyond their role, the missing BlindQA and mutual exclusion settings, and the in-house users with access they shouldn't have. what is the update on each of these from the team convos so farMar 20Status of each observation from team conversations: EU filter / content routing — Legal (Eric Brothers) has been contacted but hasn't responded yet. Expected next week. The question is whether this constitutes a breach of EU data residency requirements. Kheira flagged that we should confirm whether audio content is a permitted content type under legal's guidelines. Tim confirmed that even though violations are audio-based, the labeler still sees the full content — video, keyframes, user profile, bio, comments. We've been asked to confirm the full year numbers which are currently running. Non-public content filter — Kheira asked us to add this to the observation document, which we've done. The team aligned that we can report using Live AIM data only for now. Julia's team is running the VQA equivalent separately. Tim and Julia discussed that private accounts going live is inherently unusual so the numbers may be smaller than SFV, but the control gap is the same regardless of volume. Duplication — Vanessa (SMO stakeholder) questioned whether this could be intentional — she said they sometimes inject similar content to multiple labelers to see how the majority votes. We clarified this is different — this is a pipeline-level issue at the extraction stage, not intentional QA calibration. Adam from R&D recognized it immediately as a missing dedup function. We need to add a note in the observation explaining why this is not intentional complex case injection. QA pipeline — Overlaps with Mateja's work on the GCA side. We shared our pipeline trace data with her including queue IDs and task-level evidence. Tim wants us to align on findings before reporting to avoid duplication in the report. No pushback from stakeholders on this one yet. System access — Vanessa was surprised by the permission volumes. She said they recently did an exercise on this and didn't expect the numbers to be this high. She wants to double check. We need to confirm whether our permission data is point-in-time or reflects the full year. The table appears to show current state so the numbers represent permissions as of the query date. walk through the full injection pipeline to the quesMar 20How content gets from a livestream into a labeling queue: It starts with someone going live on TikTok. While they're streaming, AI models are running in the background scoring the audio for potential policy violations — things like hate speech, bullying, sexual content, slurs. When the model flags something above a confidence threshold, that clip gets recorded — roughly a 20-second segment — along with keyframes (still images from the video), the ASR text (speech-to-text transcript), the user's profile info (nickname, bio), comments, and the country code of where they're streaming from. This all lands in a large source table — basically a holding area of flagged livestream clips with their model scores. From there, the R&D team has extraction jobs scheduled — these are called Dorado tasks. Each one runs a SQL query against that source table saying: give me clips in Kurdish, above this model score, from the last 24 hours. The results get packaged up into a batch. That batch gets submitted through an injection service, which routes each task through a system called TikHawk. TikHawk looks at the language and policy and determines which TCS queue it belongs in. The task lands in the queue. A BPO labeler picks it up, watches the clip, reviews the keyframes and ASR text, answers the policy decision tree questions, and submits their answer. A percentage of completed tasks then get automatically sampled into a separate QA queue for blind review. If the blind reviewer disagrees with the labeler, it becomes a diff case and goes to arbitration. The final labeling decisions — from both the original labeling and any QA outcomes — feed into the model training pipeline. That's the end goal. The labels are what the AI models learn from. give aa walkthrough of eash of the observations.. give it to me in clear straight talking points.. focus on what is going wrong and what issue isMar 201. EU Content Filter Content routes to queues by language, not geography EU users streaming in non-EU languages end up in queues worked by BPO sites outside the EU A filter exists in the platform to catch this — it's working on other queues It's not turned on for any ADSO queue Roughly 1,700 to 2,400 EU-origin tasks per day being labeled outside the EU No US filter exists at all 2. Non-Public Content Filter Same story, different control Platform can auto-remove private account content before labelers see it Active on other queues, ~25,000 times per day Not turned on for any ADSO queue About 4.7% of Live AIM tasks involve private account content — roughly 921,000 tasks for the full year BPO workers are viewing and labeling private user content with no indication it's private 3. Duplication Extraction jobs pull flagged clips from a source table and push them into queues When multiple jobs run on the same day for the same queue, they pull from the same pool independently Nothing checks if a clip was already pulled by an earlier job that day Both copies go in, both get labeled by different people, both results go to model training Smaller markets get hit worst — Kurdish at 32%, Hausa at 23% Every queue is affected, ~95,000 duplicate tasks in December alone No dedup function exists anywhere in the pipeline 4. QA Pipeline Broken Three-layer QA process: labeling, blind review, arbitration Layer 3 arbitration stopped in six markets around Q3 2025 — Arabic, English, Turkish, German, French, Spanish EU. The arbitrators left and were never replaced. Tasks pile up for 30 days and expire. Layer 2 blind review stopped in four more markets — Azerbaijani, Hebrew, Swedish, Ukrainian. No reviewer is assigned. Tasks get sampled in and nobody is on the other end. Eight markets never had a QA queue at all — Chinese, Japanese, Korean, Kurdish, Nepali, Filipino, Russian, Urdu. About 350,000 tasks per month go straight to model training with zero quality review. For markets where QA does work, about 54,000 tasks expired in December before anyone got to them. Capacity doesn't match the volume being sampled. 5. System Access The permission system doesn't enforce roles. It doesn't distinguish between a labeler, a QA reviewer, and a team lead on a given queue. BlindQA setting — hides the first-round answer from the QA reviewer. Not turned on for 86% of VQA queues and 13% of Live AIM queues. QA reviewers can see the answer before they review. Mutual exclusion setting — prevents the same person from labeling and QA-reviewing the same task. Not turned on for 99% of VQA queues and 100% of Live AIM queues. Over a thousand BPO workers have permission to browse open tasks beyond their own assignments 18 BPO workers can view completed task data and historical labeling decisions across 12 Live AIM queues 20 in-house users who aren't project owners also have access to view task data they shouldn't i said explain it.. doesnt just give me random tid bitsMar 201. EU Content Filter Content gets routed to labeling queues based on language. So if someone is in Germany but streaming in Arabic, their content goes to the Arabic queue. That queue is worked by a BPO vendor in Morocco. So EU user data just left the EU without any check. The platform actually has a control to prevent this. It detects EU-origin content in non-EU queues and removes it before any labeler sees it. This control is live and working on other queues across the platform — tens of thousands of activations per month. On ADSO queues, this control has never fired. Not once. The reason is that each queue has a classification field called entity. The control only applies to entity 1 queues. All ADSO queues are entity 9. The control was never extended to cover them. So the capability exists, it's deployed, it works — it just doesn't reach the queues we're auditing. Roughly 1,700 to 2,400 EU-origin tasks per day are being labeled outside the EU as a result. We also checked whether there's an equivalent control for US content. There isn't. No US filter exists on any queue in the platform. 2. Non-Public Content Filter Same pattern as the EU filter. The platform has a control that detects content from private user accounts and removes it from queues before labelers can access it. This control is active on other queues — about 25,000 activations per day. On ADSO queues, zero activations. The control isn't turned on. So content from private accounts enters the queue and gets labeled by BPO workers like any other task. They have no way of knowing the account is private. About 4.7% of completed Live AIM tasks involved private account content. Consistent across all 31 queues. That works out to roughly 921,000 tasks for the full year. 3. Duplication To understand this you need to know how content enters the queues. The R&D team runs extraction jobs that query a source table of flagged livestream clips. Each job says give me all clips in this language, above this confidence score, from the last 24 hours. The results get packaged into a batch and injected into the queue. When one job runs per day, no problem. The clips get pulled, injected, and labeled. The issue is when multiple jobs run on the same day for the same queue. Each job queries the same source table independently. The first job pulls a set of clips. The second job runs later and queries the same table with the same filters. The clips from the first job are still sitting there — nothing marks them as already selected. So the second job pulls them again. Both copies get injected into the queue. Two different labelers get assigned what is actually the same content. They both label it. Both results go into model training. There is no deduplication check anywhere in the pipeline. Not at extraction, not at injection, not at the queue. This is a standard function you would expect in any data pipeline — you assign a unique identifier so the second batch knows to skip anything the first batch already took. That function doesn't exist here. Smaller markets get hit hardest because the content pool is smaller. Kurdish has a 32% duplication rate — nearly a third of all labeling in that market is redundant. Hausa is at 23%. Dutch at 14%. Even the largest markets have it, just at lower rates. Every single queue is affected. In December alone, about 95,000 tasks were duplicates. 4. QA Pipeline Broken Live AIM has a three-layer QA process. First layer is the labeling itself. Second layer is blind review — a percentage of completed tasks get copied to a separate QA queue where a different person labels the same content without seeing the first answer. If they disagree, it becomes a diff case and goes to the third layer — an in-house arbitrator who makes the final call on who was right. What we found is that this pipeline is broken at different points across different markets. In six markets — Arabic, English, Turkish, German, French, Spanish EU — the third layer stopped working around Q3 2025. The arbitrators who used to handle diff cases left and were never replaced. But the system doesn't know nobody is there. Diff cases keep flowing into the arbitration queue, sit there for 30 days, and get automatically discarded. We can see these queues were working earlier in the year — there are completed arbitration tasks in July, August, September. Then it stops and never recovers. In four more markets — Azerbaijani, Hebrew, Swedish, Ukrainian — it's worse. Both the second and third layers are broken. Tasks get sampled into the QA queue, the system copies in the first-round answer as a placeholder, and then nothing happens. No QA reviewer is assigned to pick them up. They sit for 30 days and expire. For these markets there is zero quality review at any layer. Then there are eight markets that never had a QA pipeline at all. No QA queue was ever created for them. Chinese, Japanese, Korean, Kurdish, Nepali, Filipino, Russian, Urdu. Content gets labeled and goes straight to model training with no independent check. These aren't small markets either — combined they account for about 350,000 tasks per month. And even for the markets where QA is functioning, a significant portion of sampled tasks are expiring before anyone reviews them. About 54,000 QA tasks expired in December. PT EU lost 67% of its QA tasks to expiry. IT lost 41%. The review capacity simply isn't matched to the volume being sampled. So across the board, the documented QA process is either not in place, not staffed, or not keeping up. 5. System Access The way the permission system works is that users get assigned to queues with general permissions — things like read, write, export. But the system doesn't distinguish between roles. It doesn't know whether you're a labeler, a QA reviewer, or a team lead on that queue. Everyone with access to the queue looks the same to the system. To compensate for this, there are two queue-level settings. The first is BlindQA — when turned on, the QA reviewer can't see what the first-round labeler decided. This is the whole point of blind review — the reviewer forms their own judgment independently. When it's off, the reviewer sees the answer before they review, which undermines the independence of the check. For VQA, 86% of queues have this turned off. For Live AIM, 13%. The second is mutual exclusion — when turned on, the system prevents the same person from labeling a task and then QA-reviewing that same task. When it's off, nothing stops someone from reviewing their own work. For VQA, 99% of queues have this off. For Live AIM, 100%. On top of the configuration issues, we looked at what permissions have actually been granted to users. BPO labelers should only have access to complete their own assigned tasks. We found over a thousand BPO workers across both projects with a permission called "search incompleted tasks" — this lets them browse all open tasks in the queue, including other people's work and tasks awaiting QA. There's no reason for a labeler to see any of that. We also found 18 BPO workers with "view task data" permission across 12 Live AIM queues. This gives them access to completed labeling decisions and historical task data — well beyond what a labeler needs. Same issue on the in-house side. 17 in-house users on VQA and 3 on Live AIM have "view task data" permission but aren't project POCs. Only project owners should have this level of visibility. The bottom line is that the system doesn't enforce role-based access. The settings that would enforce separation between labeling and QA are mostly turned off, and permissions have been granted beyond what each role requires. summaryize the whole process end to end in one sentenceMar 20TikTok's AI models flag livestream content for potential policy violations, that content gets extracted and injected into language-based labeling queues where third-party BPO workers review it and submit moderation decisions, which then feed into the model training pipeline to improve the AI.