I am importing a CSV file into a temporary table and then running a select query that joins my actual database table with the temporary table, looking for any changes in the data. If changes exist, the select query is looped from 1 to #recordCount# and an update is applied to each record via cfquery. It runs very quickly (much more quickly than looping the query itself), but my memory spikes and overloads after about 1500 updates are completed. I need to be able to do upwards of 20000 at a time, without killing my system. I have tried manually setting the runtime garbage collection to trigger after X number of loops, but that doesn't seem to help. I am running CF8. See below for loop example:
<cfloop from="1" to="#updatedRecordsQuery.recordCount#" index="a">
<cftry>
<cfquery datasource="#db#" name="doUpdate">
UPDATE
CI
SET
firstname = <cfqueryparam cfsqltype="cf_sql_varchar" value="#updatedRecordsQuery.firstname[a]#" />,
lastname = <cfqueryparam cfsqltype="cf_sql_varchar" value="#updatedRecordsQuery.lastname[a]#" />,
etc, for about 15 various fields
FROM
client_info CI
WHERE
CI.client_id = <cfqueryparam cfsqltype="cf_sql_integer" value="#updatedRecordsQuery.client_id[a]#" />
</cfquery>
<cfcatch type="database">
<cfset local.updateErrorList = listappend(local.updateErrorList,updatedRecordsQuery.client_id[a]) />
<cfset local.error = true />
</cfcatch>
</cftry>
</cfloop>